Sep 09 03:11:16.224606 ip-10-0-142-178 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 03:11:16.224621 ip-10-0-142-178 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 03:11:16.224631 ip-10-0-142-178 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 03:11:16.224963 ip-10-0-142-178 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 03:11:25.874793 ip-10-0-142-178 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 09 03:11:25.874810 ip-10-0-142-178 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a739a280fc464c9c8e6cff0ab0939cb4 --
Sep 09 03:13:09.172978 ip-10-0-142-178 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 03:13:09.611121 ip-10-0-142-178 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:13:09.611121 ip-10-0-142-178 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 03:13:09.611121 ip-10-0-142-178 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:13:09.611121 ip-10-0-142-178 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 03:13:09.611121 ip-10-0-142-178 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:13:09.614061 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.613991    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 03:13:09.620907 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620890    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:09.620907 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620905    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:09.620907 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620909    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620912    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620916    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620919    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620921    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620924    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620927    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620929    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620932    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620934    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620937    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620939    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620943    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620946    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620948    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620951    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620953    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620956    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620959    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620961    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:09.621014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620964    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620966    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620969    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620971    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620973    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620976    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620978    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620981    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620983    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620986    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620988    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620990    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620993    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.620999    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621003    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621007    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621011    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621014    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621017    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:09.621490 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621021    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621024    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621026    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621029    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621032    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621035    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621038    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621041    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621044    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621046    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621048    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621051    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621054    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621056    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621059    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621061    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621064    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621066    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621069    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:09.622004 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621071    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621074    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621076    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621079    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621082    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621084    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621087    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621090    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621093    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621096    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621099    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621101    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621104    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621107    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621109    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621112    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621115    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621117    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621120    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:09.622478 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621123    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621125    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621127    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621130    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621132    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621136    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621138    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621141    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621144    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621146    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621148    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621151    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621153    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621156    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621159    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621163    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621165    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621168    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621171    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621173    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:09.622956 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621873    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621880    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621884    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621886    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621889    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621892    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621894    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621897    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621900    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621902    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621905    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621907    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621910    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621913    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621916    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621920    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621923    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621926    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621929    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:09.623439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621932    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621935    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621938    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621940    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621943    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621946    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621948    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621951    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621953    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621956    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621958    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621961    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621963    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621966    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621969    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621971    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621974    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621977    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621981    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:09.623927 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621984    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621987    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621989    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621992    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621994    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.621997    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622000    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622008    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622011    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622013    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622016    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622019    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622021    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622024    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622027    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622029    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622031    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622034    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622036    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:09.624475 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622039    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622041    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622044    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622046    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622048    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622050    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622053    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622056    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622058    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622060    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622063    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622065    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622068    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622070    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622073    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622075    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622077    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622080    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622082    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622084    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:09.624995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622087    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622090    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622093    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622095    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622097    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622101    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622104    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622106    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622109    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622111    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622114    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622116    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622119    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622121    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622123    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622126    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622128    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622130    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622135    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622137    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:09.625473 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622140    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622143    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622550    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622555    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622559    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622562    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622565    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622568    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622571    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622573    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622576    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622579    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622582    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622586    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622589    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622591    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622594    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622597    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622599    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622602    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:09.625979 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622604    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622607    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622609    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622612    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622614    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622616    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622619    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622621    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622624    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622626    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622630    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622633    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622635    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622638    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622640    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622642    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622645    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622647    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622650    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622653    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:09.626486 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622655    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622658    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622660    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622663    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622665    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622682    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622686    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622688    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622691    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622694    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622696    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622699    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622701    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622704    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622706    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622709    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622711    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622714    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622716    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622720    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:09.627016 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622724    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622726    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622730    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622733    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622737    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622741    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622744    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622746    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622749    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622752    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622755    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622758    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622761    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622764    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622767    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622770    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622772    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622776    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622778    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:09.627511 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622781    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622783    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622786    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622788    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622791    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622793    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622796    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622798    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622801    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622803    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622806    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622808    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622811    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622813    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622815    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622819    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622821    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622824    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622826    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622829    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:09.628001 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622831    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.622834    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622897    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622928    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622934    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622939    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622942    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622945    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622949    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622953    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622958    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622961    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622964    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622968    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622971    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622974    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622976    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622980    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622983    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622986    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622989    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622992    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622995    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.622998    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 03:13:09.628487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623001    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623004    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623007    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623011    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623014    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623017    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623020    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623023    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623026    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623029    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623032    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623035    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623038    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623041    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623044    2580 flags.go:64] FLAG: --enable-server="true"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623046    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623051    2580 flags.go:64] FLAG: --event-burst="100"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623053    2580 flags.go:64] FLAG: --event-qps="50"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623057    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623060    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623063    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623066    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623070    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623073    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623075    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 09 03:13:09.629068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623079    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623082    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623084    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623087    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623090    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623093    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623096    2580 flags.go:64] FLAG: --feature-gates=""
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623099    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623102    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623105    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623109    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623112    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623115    2580 flags.go:64] FLAG: --help="false"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623118    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623121    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623124    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623127    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623130    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623133    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623136    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623139    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623141    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623144    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 03:13:09.629733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623147    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623150    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623153    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623156    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623159    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623162    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623164    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623168    2580 flags.go:64] FLAG: --lock-file=""
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623171    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623173    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623176    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623181    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623184    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623186    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623189    2580 flags.go:64] FLAG: --logging-format="text"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623192    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623195    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623198    2580 flags.go:64] FLAG: --manifest-url=""
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623200    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623206    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623208    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623212    2580 flags.go:64] FLAG: --max-pods="110"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623215    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623218    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623221    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 03:13:09.630439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623224    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623226    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623229    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623232    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623238    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623241    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623244    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623247    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623250    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623255    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623259    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623262    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623265    2580 flags.go:64] FLAG: --port="10250"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623268    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623271    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00733dc1f10495943"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623274    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623277    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623280    2580 flags.go:64] FLAG: --register-node="true"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623283    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623286    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623289    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623291    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623294    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623298    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623300    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 03:13:09.631402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623303    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623308    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623311    2580 flags.go:64] FLAG: --runonce="false"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623313    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623316    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623319    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623322    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623325    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623327    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623330    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623333    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623336    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623338    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623341    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623344    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623347    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623349    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623353    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623359    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623362    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623365    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623369    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623371    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623374    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623377    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 03:13:09.632075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623380    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623383    2580 flags.go:64] FLAG: --v="2"
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623386    2580 flags.go:64] FLAG: --version="false"
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623390    2580 flags.go:64] FLAG: --vmodule=""
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623393    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623396    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623502    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623505    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623509    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623512    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623514    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623517    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623519    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623522    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623525    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623528    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623530    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623533    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623536    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623538    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623541    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:09.632707 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623543    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623546    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623548    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623552    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623554    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623557    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623559    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623562    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623564    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623567    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623572    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623575    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623577    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623580    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623582    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623586    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623590    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623593    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623596    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:09.633231 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623601    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623604    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623607    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623610    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623612    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623614    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623617    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623619    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623622    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623624    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623627    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623630    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623632    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623635    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623637    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623639    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623643    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623645    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623648    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623650    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:09.633730 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623653    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623655    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623657    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623660    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623663    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623665    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623684    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623686    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623689    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623692    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623694    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623697    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623701    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623704    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623706    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623709    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623712    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623714    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623717    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623719    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:09.634227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623722    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623726    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623729    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623732    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623735    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623737    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623747    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623749    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623753    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623756    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623758    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623761    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623764    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623766    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623769    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623771    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623774    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623777    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623779    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:09.634787 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623782    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623784    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623787    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623789    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623792    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.623795    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.623801    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.632198    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.632213    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.632232    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.632238    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632292    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632297    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632300    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632303    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:09.635239 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632306    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632309    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632312    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632314    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632317    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632320    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632322    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632325    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632327    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632330    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632332    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632335    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632337    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632340    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632342    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632344    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632347    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632349    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632353    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:09.635602 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632355    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632358    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632361    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632363    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632365    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632368    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632371    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632374    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632377    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632379    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632382    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632384    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632387    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632389    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632391    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632394    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632396    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632399    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632401    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632403    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:09.636154 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632406    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632408    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632411    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632413    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632416    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632419    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632421    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632423    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632426    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632428    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632431    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632434    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632436    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632440    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632443    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632447    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632451    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632455    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632458    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:09.636634 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632461    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632464    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632466    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632468    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632471    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632473    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632476    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632478    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632481    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632483    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632486    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632488    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632490    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632493    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632495    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632498    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632500    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632503    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632505    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:09.637124 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632508    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632510    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632513    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632515    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632517    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632520    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632522    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632525    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632527    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632529    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632532    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632534    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632537    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632540    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632542    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632545    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632549    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:09.637581 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632552    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.632557    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632662    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632685    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632689    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632692    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632695    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632698    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632701    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632703    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632706    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632708    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632711    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632714    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:13:09.638053 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632717    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632721    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632724    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632727    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632729    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632732    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632734    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632737    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632739    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632742    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632744    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632746    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632749    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632752    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632754    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632757    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632759    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632762    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632764    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632767    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:13:09.638402 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632769    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632772    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632774    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632778    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632781    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632784    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632788    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632790    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632793    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632796    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632798    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632801    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632803    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632806    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632808    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632811    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632813    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632816    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632818    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:13:09.638899 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632821    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632823    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632825    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632828    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632830    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632833    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632836    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632838    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632840    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632843    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632845    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632847    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632850    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632852    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632854    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632857    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632859    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632862    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632864    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632867    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:13:09.639354 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632869    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632871    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632873    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632876    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632878    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632881    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632883    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632886    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632889    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632892    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632894    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632896    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632899    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632901    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632904    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632906    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632908    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632911    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632913    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632916    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:13:09.639848 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632918    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632921    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632923    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632926    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632928    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632931    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632933    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:09.632936    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.632940    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.633046    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.636216    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.637339    2580 server.go:1019] "Starting client certificate rotation"
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.637443    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 03:13:09.640318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.637470    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 03:13:09.641183 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.641170    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 03:13:09.646183 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.646166    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 03:13:09.658053 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.658037    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 09 03:13:09.662067 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.662051    2580 log.go:25] "Validated CRI v1 image API"
Sep 09 03:13:09.662956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.662942    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 03:13:09.664442 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.664426    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 03:13:09.666340 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.666321    2580 fs.go:135] Filesystem UUIDs: map[6cc4b6b9-8208-45ad-8a67-99e47ae6b5ff:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c087c4c8-8b21-4469-9897-841e39e55120:/dev/nvme0n1p3]
Sep 09 03:13:09.666433 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.666340    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 03:13:09.671559 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.671452    2580 manager.go:216] Machine: {Timestamp:2026-09-09 03:13:09.670972183 +0000 UTC m=+0.370736135 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3134348 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29fa3562d300c18da7240f024d0b23 SystemUUID:ec29fa35-62d3-00c1-8da7-240f024d0b23 BootID:a739a280-fc46-4c9c-8e6c-ff0ab0939cb4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:19:b4:17:bc:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:19:b4:17:bc:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:a9:df:6c:9f:fe Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 03:13:09.671559 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.671549    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 03:13:09.671764 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.671692    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 03:13:09.674139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.674113    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 03:13:09.674276 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.674140    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 03:13:09.674947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.674936    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 03:13:09.674988 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.674950    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 03:13:09.674988 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.674958    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 03:13:09.674988 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.674975    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 03:13:09.676640 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.676629    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 03:13:09.679939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.679928    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 03:13:09.680235 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680173    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 03:13:09.680346 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680272    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 03:13:09.680346 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680288    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 03:13:09.680346 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680310    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 03:13:09.680346 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680324    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 09 03:13:09.680346 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680337    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 03:13:09.680797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680781    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 03:13:09.680872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.680806    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 03:13:09.683774 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.683752    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 03:13:09.684166 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.684146    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pk9gw"
Sep 09 03:13:09.686820 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.686807    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 03:13:09.686860 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.686837    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 03:13:09.687003 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.686992    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 03:13:09.687046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687021    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 03:13:09.687046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687032    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 03:13:09.687046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687041    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 03:13:09.687142 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687052    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 03:13:09.687142 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687058    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 03:13:09.687586 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687578    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 03:13:09.687586 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687586    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 03:13:09.687648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687593    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 03:13:09.687648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687598    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 03:13:09.687648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687606    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 03:13:09.687648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.687613    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 03:13:09.688551 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.688530    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 03:13:09.688615 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.688531    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 03:13:09.688768 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.688758    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 03:13:09.688768 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.688769    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 03:13:09.689361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.689345    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pk9gw"
Sep 09 03:13:09.692300 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.692288    2580 server.go:1268] "Started kubelet"
Sep 09 03:13:09.692419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.692366    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 03:13:09.692462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.692440    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 03:13:09.692954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.692922    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 03:13:09.693067 ip-10-0-142-178 systemd[1]: Started Kubernetes Kubelet.
Sep 09 03:13:09.694956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.694942    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 03:13:09.695986 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.695969    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 03:13:09.698183 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.698141    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 03:13:09.699456 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.699442    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 03:13:09.699561 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.699522    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 03:13:09.699625 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.699561    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:09.700157 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700138    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 03:13:09.700250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700145    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 03:13:09.700320 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700257    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 03:13:09.700392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700381    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 03:13:09.700443 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700393    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 03:13:09.700492 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700452    2580 factory.go:55] Registering systemd factory
Sep 09 03:13:09.700492 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700463    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 09 03:13:09.700645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700631    2580 factory.go:153] Registering CRI-O factory
Sep 09 03:13:09.700707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700648    2580 factory.go:223] Registration of the crio container factory successfully
Sep 09 03:13:09.700742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700735    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 03:13:09.700777 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700750    2580 factory.go:103] Registering Raw factory
Sep 09 03:13:09.700777 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.700763    2580 manager.go:1196] Started watching for new ooms in manager
Sep 09 03:13:09.701086 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.701074    2580 manager.go:349] Starting recovery of all containers
Sep 09 03:13:09.701804 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.701778    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:09.708261 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.708239    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-178.ec2.internal" not found
Sep 09 03:13:09.708261 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.708246    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-178.ec2.internal\" not found" node="ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.715195 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.715179    2580 manager.go:354] Recovery completed
Sep 09 03:13:09.719180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.719167    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.720647 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.720630    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.720743 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.720658    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.720743 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.720703    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.721200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721188    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 03:13:09.721200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721199    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 03:13:09.721272 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721206    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.721272 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721215    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 03:13:09.721909 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721893    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.721982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721918    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.721982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.721928    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.722280 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.722268    2580 policy_none.go:49] "None policy: Start"
Sep 09 03:13:09.722317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.722284    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 03:13:09.722317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.722294    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 03:13:09.723112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.723101    2580 policy_none.go:47] "Start"
Sep 09 03:13:09.725100 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.725086    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-178.ec2.internal" not found
Sep 09 03:13:09.760800 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.760787    2580 manager.go:337] "Starting Device Plugin manager"
Sep 09 03:13:09.760870 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.760818    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 03:13:09.760870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.760827    2580 server.go:87] "Starting device plugin registration server"
Sep 09 03:13:09.761067 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.761054    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 03:13:09.761102 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.761081    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 03:13:09.761174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.761155    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 03:13:09.761353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.761237    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 03:13:09.761353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.761248    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 03:13:09.761737 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.761715    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 03:13:09.761837 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.761771    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:09.780967 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.780947    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 03:13:09.782264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.782246    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 03:13:09.782352 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.782288    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 03:13:09.782352 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.782311    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 03:13:09.782352 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.782345    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 03:13:09.784471 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.784450    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-178.ec2.internal" not found
Sep 09 03:13:09.784540 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.784483    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:09.861611 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.861544    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.862300 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.862287    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.862381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.862314    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.862381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.862328    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.862381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.862355    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.871902 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.871887    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.871980 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.871906    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-178.ec2.internal\": node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:09.882639 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.882616    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"]
Sep 09 03:13:09.882741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.882699    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.883988 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.883974    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.884069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.884002    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.884069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.884015    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.884960 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.884946    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.885081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.885129 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885093    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.885576 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885560    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.885637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885588    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.885637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885597    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.885717 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885560    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.885717 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885686    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.885717 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.885704    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.886836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.886822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.886897 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.886846    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:13:09.887496 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.887480    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:13:09.887554 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.887514    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:13:09.887554 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.887528    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:13:09.893328 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.893310    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:09.902560 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.902540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b9765c33d70e82b31ab7040648baa6e-config\") pod \"kube-apiserver-proxy-ip-10-0-142-178.ec2.internal\" (UID: \"8b9765c33d70e82b31ab7040648baa6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.902650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.902581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bdb911bab22f2a07de1ddc19e29a1e21-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal\" (UID: \"bdb911bab22f2a07de1ddc19e29a1e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.902650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:09.902598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdb911bab22f2a07de1ddc19e29a1e21-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal\" (UID: \"bdb911bab22f2a07de1ddc19e29a1e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.914267 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.914250    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-178.ec2.internal\" not found" node="ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.918584 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.918567    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-178.ec2.internal\" not found" node="ip-10-0-142-178.ec2.internal"
Sep 09 03:13:09.994403 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:09.994375    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.003061 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.003022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b9765c33d70e82b31ab7040648baa6e-config\") pod \"kube-apiserver-proxy-ip-10-0-142-178.ec2.internal\" (UID: \"8b9765c33d70e82b31ab7040648baa6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.003061 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.003046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bdb911bab22f2a07de1ddc19e29a1e21-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal\" (UID: \"bdb911bab22f2a07de1ddc19e29a1e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.003184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.003063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdb911bab22f2a07de1ddc19e29a1e21-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal\" (UID: \"bdb911bab22f2a07de1ddc19e29a1e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.003184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.003120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdb911bab22f2a07de1ddc19e29a1e21-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal\" (UID: \"bdb911bab22f2a07de1ddc19e29a1e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.003184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.003127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b9765c33d70e82b31ab7040648baa6e-config\") pod \"kube-apiserver-proxy-ip-10-0-142-178.ec2.internal\" (UID: \"8b9765c33d70e82b31ab7040648baa6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.003184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.003141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bdb911bab22f2a07de1ddc19e29a1e21-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal\" (UID: \"bdb911bab22f2a07de1ddc19e29a1e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.094727 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.094712    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.195466 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.195421    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.217880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.217860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.222723 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.222709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:10.295971 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.295944    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.396493 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.396472    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.497000 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.496944    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.597512 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.597487    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.611686 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.611654    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:10.636950 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.636804    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 03:13:10.637094 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.637075    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:13:10.637149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.637091    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:13:10.637149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.637098    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:13:10.694300 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.694264    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 03:08:10 +0000 UTC" deadline="2028-02-28 07:15:38.321019207 +0000 UTC"
Sep 09 03:13:10.694300 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.694298    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12892h2m27.62672388s"
Sep 09 03:13:10.698420 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.698402    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.734139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.734125    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:13:10.785305 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.785237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal" event={"ID":"8b9765c33d70e82b31ab7040648baa6e","Type":"ContainerStarted","Data":"1cfd410674b99d9bd7b555faf949dc86f79a14eaa4246993d14529c7b2593fab"}
Sep 09 03:13:10.786236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:10.786214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal" event={"ID":"bdb911bab22f2a07de1ddc19e29a1e21","Type":"ContainerStarted","Data":"234444a99d956ecc844a5e6c4f5fdd64830c92609b5ef3ee2f4c6ccd8e716cde"}
Sep 09 03:13:10.799493 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.799479    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:10.899901 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:10.899881    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:11.000310 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.000290    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:11.100738 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.100691    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:11.201515 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.201484    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-178.ec2.internal\" not found"
Sep 09 03:13:11.211069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.211042    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:11.300039 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.300006    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:11.307266 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.307241    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 03:13:11.308403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.308370    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal"
Sep 09 03:13:11.317249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.317223    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 03:13:11.566178 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.566149    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:11.683271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.683199    2580 apiserver.go:52] "Watching apiserver"
Sep 09 03:13:11.691317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.691292    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 03:13:11.692607 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.692582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467","openshift-cluster-node-tuning-operator/tuned-sqnst","openshift-dns/node-resolver-kdgpc","openshift-image-registry/node-ca-zrhtb","openshift-multus/multus-lgf6v","openshift-network-diagnostics/network-check-target-n498h","kube-system/konnectivity-agent-jwkkl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal","openshift-multus/multus-additional-cni-plugins-rmm89","openshift-multus/network-metrics-daemon-fb6s8","openshift-network-operator/iptables-alerter-fv2sz","openshift-ovn-kubernetes/ovnkube-node-n5sjp","kube-system/global-pull-secret-syncer-tx8l4","kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal"]
Sep 09 03:13:11.694071 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.694048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.695260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.695168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.695260 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.695252    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:11.696228 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.696193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.696623 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.696605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b9mlp\""
Sep 09 03:13:11.696725 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.696650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 03:13:11.696943 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.696927    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 03:13:11.697813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.697789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.698473 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.698453    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.698582 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.698512    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.698705 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.698671    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hrwmn\""
Sep 09 03:13:11.698930 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.698915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.699809 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.699671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:11.699809 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.699768    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:11.699977 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.699832    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 03:13:11.700277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.700259    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.700485 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.700462    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q7lsd\""
Sep 09 03:13:11.700595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.700523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.700595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.700569    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 03:13:11.701162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.701121    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.701162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.701153    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 03:13:11.701524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.701505    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.701606 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.701556    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tz2vp\""
Sep 09 03:13:11.701606 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.701508    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 03:13:11.701714 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.701608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.702977 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.702957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.703941 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.703803    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 03:13:11.703941 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.703914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 03:13:11.704132 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.704112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kfxjx\""
Sep 09 03:13:11.705303 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.704865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.708433 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.708097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:11.708433 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.708171    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:11.708433 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.708341    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.708433 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.708378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.708701 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.708436    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tt6wv\""
Sep 09 03:13:11.708872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.708848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 03:13:11.709229 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.709163    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kdfzf\""
Sep 09 03:13:11.710222 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.709817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.710541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.710521    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.710726 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.710694    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.711419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.711809 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.711928 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-system-cni-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.711928 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-cni-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.711928 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-socket-dir-parent\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-k8s-cni-cncf-io\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.711979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:11.712081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99922913-bda7-4f39-9dbe-ee67f87431f9-serviceca\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.712081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-cnibin\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-kubelet\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-hostroot\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-dbus\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-cni-bin\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b94zd\" (UniqueName: \"kubernetes.io/projected/99922913-bda7-4f39-9dbe-ee67f87431f9-kube-api-access-b94zd\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxzn\" (UniqueName: \"kubernetes.io/projected/96ed75e1-217e-49b7-939a-07720b22def2-kube-api-access-mcxzn\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db67cce9-62c6-45b2-b7af-24895bd4519b-cni-binary-copy\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-netns\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-conf-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-etc-kubernetes\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca41c40-7763-4d20-9e3a-6fd665bccb52-tmp-dir\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-multus-certs\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bad8da5-211c-4a2a-adc6-130d5a3e7d9b-konnectivity-ca\") pod \"konnectivity-agent-jwkkl\" (UID: \"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b\") " pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-kubelet-config\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-cni-binary-copy\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bad8da5-211c-4a2a-adc6-130d5a3e7d9b-agent-certs\") pod \"konnectivity-agent-jwkkl\" (UID: \"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b\") " pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ca41c40-7763-4d20-9e3a-6fd665bccb52-hosts-file\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99922913-bda7-4f39-9dbe-ee67f87431f9-host\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-daemon-config\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k476f\""
Sep 09 03:13:11.712685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v27hp\" (UniqueName: \"kubernetes.io/projected/1ca41c40-7763-4d20-9e3a-6fd665bccb52-kube-api-access-v27hp\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.713337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-system-cni-dir\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.713337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-cnibin\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.713337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-cni-multus\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.713337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blklm\" (UniqueName: \"kubernetes.io/projected/db67cce9-62c6-45b2-b7af-24895bd4519b-kube-api-access-blklm\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.713337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712890    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.713337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.712934    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.713538 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.713525    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 03:13:11.713587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.713552    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 03:13:11.713833 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.713811    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 03:13:11.713907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.713836    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 03:13:11.714174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.714117    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 03:13:11.715316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.715104    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 03:13:11.715316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.715225    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z4h5g\""
Sep 09 03:13:11.715590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.715569    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 03:13:11.738814 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.738789    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9dbm5"
Sep 09 03:13:11.746708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.746690    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9dbm5"
Sep 09 03:13:11.777477 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.777451    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:11.801016 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.800995    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 03:13:11.813205 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-cnibin\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.813308 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-kubelet\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813308 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-hostroot\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813308 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-modprobe-d\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.813427 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-cnibin\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.813427 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-kubelet\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-hostroot\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-dbus\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.813532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-cni-bin\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-env-overrides\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.813662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-systemd\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.813662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-dbus\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.813662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b94zd\" (UniqueName: \"kubernetes.io/projected/99922913-bda7-4f39-9dbe-ee67f87431f9-kube-api-access-b94zd\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.813662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-netns\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-conf-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdcbg\" (UniqueName: \"kubernetes.io/projected/20739dea-771d-479e-9846-0f88eba250b8-kube-api-access-mdcbg\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-netns\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysctl-conf\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-conf-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-cni-bin\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca41c40-7763-4d20-9e3a-6fd665bccb52-tmp-dir\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-ovnkube-config\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bad8da5-211c-4a2a-adc6-130d5a3e7d9b-konnectivity-ca\") pod \"konnectivity-agent-jwkkl\" (UID: \"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b\") " pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.813939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-kubelet\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-run-netns\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.813991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-ovnkube-script-lib\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj8js\" (UniqueName: \"kubernetes.io/projected/27730a13-1bd3-4f06-a6cb-cd260ebc137b-kube-api-access-hj8js\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysconfig\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bad8da5-211c-4a2a-adc6-130d5a3e7d9b-agent-certs\") pod \"konnectivity-agent-jwkkl\" (UID: \"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b\") " pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99922913-bda7-4f39-9dbe-ee67f87431f9-host\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-daemon-config\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pssg9\" (UniqueName: \"kubernetes.io/projected/5adb8729-7a58-4cba-9f38-ea9ce848593e-kube-api-access-pssg9\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-cni-netd\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814146    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ca41c40-7763-4d20-9e3a-6fd665bccb52-tmp-dir\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27730a13-1bd3-4f06-a6cb-cd260ebc137b-host-slash\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-sys-fs\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.814462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v27hp\" (UniqueName: \"kubernetes.io/projected/1ca41c40-7763-4d20-9e3a-6fd665bccb52-kube-api-access-v27hp\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-cnibin\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-cni-multus\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blklm\" (UniqueName: \"kubernetes.io/projected/db67cce9-62c6-45b2-b7af-24895bd4519b-kube-api-access-blklm\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20739dea-771d-479e-9846-0f88eba250b8-ovn-node-metrics-cert\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bad8da5-211c-4a2a-adc6-130d5a3e7d9b-konnectivity-ca\") pod \"konnectivity-agent-jwkkl\" (UID: \"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b\") " pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-socket-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1972d46b-dd39-454c-b163-c2aa32794edf-etc-tuned\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-cnibin\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-socket-dir-parent\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99922913-bda7-4f39-9dbe-ee67f87431f9-host\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-var-lib-cni-multus\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-k8s-cni-cncf-io\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814808    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-daemon-config\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-socket-dir-parent\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.815204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-device-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-kubernetes\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-k8s-cni-cncf-io\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-host\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814938    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.814931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99922913-bda7-4f39-9dbe-ee67f87431f9-serviceca\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-lib-modules\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-var-lib-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-log-socket\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysctl-d\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-run\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxzn\" (UniqueName: \"kubernetes.io/projected/96ed75e1-217e-49b7-939a-07720b22def2-kube-api-access-mcxzn\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db67cce9-62c6-45b2-b7af-24895bd4519b-cni-binary-copy\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-etc-kubernetes\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-slash\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-node-log\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.816029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/27730a13-1bd3-4f06-a6cb-cd260ebc137b-iptables-alerter-script\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-etc-kubernetes\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-var-lib-kubelet\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/99922913-bda7-4f39-9dbe-ee67f87431f9-serviceca\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-multus-certs\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-host-run-multus-certs\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-systemd-units\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-cni-bin\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1972d46b-dd39-454c-b163-c2aa32794edf-tmp\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5psz\" (UniqueName: \"kubernetes.io/projected/1972d46b-dd39-454c-b163-c2aa32794edf-kube-api-access-n5psz\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-kubelet-config\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-cni-binary-copy\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db67cce9-62c6-45b2-b7af-24895bd4519b-cni-binary-copy\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-registration-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-kubelet-config\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:11.816817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ca41c40-7763-4d20-9e3a-6fd665bccb52-hosts-file\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ca41c40-7763-4d20-9e3a-6fd665bccb52-hosts-file\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.815795    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-etc-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.815933    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:12.315891681 +0000 UTC m=+3.015655649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-ovn\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.815999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-system-cni-dir\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-sys\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-system-cni-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-cni-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ed75e1-217e-49b7-939a-07720b22def2-system-cni-dir\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.817463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:11.818130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-multus-cni-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.818130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-systemd\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.818130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw86x\" (UniqueName: \"kubernetes.io/projected/6809c636-8293-47c3-b326-7b09c2502d4c-kube-api-access-xw86x\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.818130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db67cce9-62c6-45b2-b7af-24895bd4519b-system-cni-dir\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.818130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-cni-binary-copy\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.818130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.816855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96ed75e1-217e-49b7-939a-07720b22def2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.819926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.819901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bad8da5-211c-4a2a-adc6-130d5a3e7d9b-agent-certs\") pod \"konnectivity-agent-jwkkl\" (UID: \"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b\") " pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:11.822435 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.822417    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:11.822532 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.822438    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:11.822532 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.822452    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:11.822532 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.822524    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:12.32250852 +0000 UTC m=+3.022272464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:11.822816 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.822794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b94zd\" (UniqueName: \"kubernetes.io/projected/99922913-bda7-4f39-9dbe-ee67f87431f9-kube-api-access-b94zd\") pod \"node-ca-zrhtb\" (UID: \"99922913-bda7-4f39-9dbe-ee67f87431f9\") " pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:11.823734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.823713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blklm\" (UniqueName: \"kubernetes.io/projected/db67cce9-62c6-45b2-b7af-24895bd4519b-kube-api-access-blklm\") pod \"multus-lgf6v\" (UID: \"db67cce9-62c6-45b2-b7af-24895bd4519b\") " pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:11.824038 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.824016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v27hp\" (UniqueName: \"kubernetes.io/projected/1ca41c40-7763-4d20-9e3a-6fd665bccb52-kube-api-access-v27hp\") pod \"node-resolver-kdgpc\" (UID: \"1ca41c40-7763-4d20-9e3a-6fd665bccb52\") " pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:11.824656 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.824631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxzn\" (UniqueName: \"kubernetes.io/projected/96ed75e1-217e-49b7-939a-07720b22def2-kube-api-access-mcxzn\") pod \"multus-additional-cni-plugins-rmm89\" (UID: \"96ed75e1-217e-49b7-939a-07720b22def2\") " pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:11.917242 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-modprobe-d\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.917377 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-env-overrides\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917377 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-systemd\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.917495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-modprobe-d\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.917495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdcbg\" (UniqueName: \"kubernetes.io/projected/20739dea-771d-479e-9846-0f88eba250b8-kube-api-access-mdcbg\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysctl-conf\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-systemd\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-ovnkube-config\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-kubelet\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-run-netns\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-kubelet\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.917650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-ovnkube-script-lib\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj8js\" (UniqueName: \"kubernetes.io/projected/27730a13-1bd3-4f06-a6cb-cd260ebc137b-kube-api-access-hj8js\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-run-netns\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysctl-conf\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysconfig\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pssg9\" (UniqueName: \"kubernetes.io/projected/5adb8729-7a58-4cba-9f38-ea9ce848593e-kube-api-access-pssg9\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-cni-netd\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysconfig\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-env-overrides\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-cni-netd\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27730a13-1bd3-4f06-a6cb-cd260ebc137b-host-slash\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-sys-fs\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917946    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27730a13-1bd3-4f06-a6cb-cd260ebc137b-host-slash\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20739dea-771d-479e-9846-0f88eba250b8-ovn-node-metrics-cert\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.917993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-socket-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-sys-fs\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.918014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1972d46b-dd39-454c-b163-c2aa32794edf-etc-tuned\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-ovnkube-config\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-socket-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-device-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/20739dea-771d-479e-9846-0f88eba250b8-ovnkube-script-lib\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-kubernetes\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-host\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-device-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-lib-modules\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-host\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-var-lib-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-kubernetes\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-var-lib-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-log-socket\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-lib-modules\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysctl-d\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-run\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-log-socket\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.918914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-slash\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-node-log\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-etc-sysctl-d\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/27730a13-1bd3-4f06-a6cb-cd260ebc137b-iptables-alerter-script\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-slash\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918477    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-node-log\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-run\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-var-lib-kubelet\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-systemd-units\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-cni-bin\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1972d46b-dd39-454c-b163-c2aa32794edf-tmp\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-systemd-units\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-cni-bin\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5psz\" (UniqueName: \"kubernetes.io/projected/1972d46b-dd39-454c-b163-c2aa32794edf-kube-api-access-n5psz\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-registration-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.919661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-var-lib-kubelet\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.918771    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:11.918824    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:13:12.418806242 +0000 UTC m=+3.118570198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-etc-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-registration-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-ovn\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918937    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-etc-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-sys\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-openvswitch\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-systemd\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.918981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-ovn\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw86x\" (UniqueName: \"kubernetes.io/projected/6809c636-8293-47c3-b326-7b09c2502d4c-kube-api-access-xw86x\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.920331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1972d46b-dd39-454c-b163-c2aa32794edf-sys\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.921146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6809c636-8293-47c3-b326-7b09c2502d4c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:11.921146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/20739dea-771d-479e-9846-0f88eba250b8-run-systemd\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.921146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.919184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/27730a13-1bd3-4f06-a6cb-cd260ebc137b-iptables-alerter-script\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.921146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.920724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1972d46b-dd39-454c-b163-c2aa32794edf-etc-tuned\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.921146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.920829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/20739dea-771d-479e-9846-0f88eba250b8-ovn-node-metrics-cert\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.921146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.921128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1972d46b-dd39-454c-b163-c2aa32794edf-tmp\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.926446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.926137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj8js\" (UniqueName: \"kubernetes.io/projected/27730a13-1bd3-4f06-a6cb-cd260ebc137b-kube-api-access-hj8js\") pod \"iptables-alerter-fv2sz\" (UID: \"27730a13-1bd3-4f06-a6cb-cd260ebc137b\") " pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:11.926446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.926404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdcbg\" (UniqueName: \"kubernetes.io/projected/20739dea-771d-479e-9846-0f88eba250b8-kube-api-access-mdcbg\") pod \"ovnkube-node-n5sjp\" (UID: \"20739dea-771d-479e-9846-0f88eba250b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:11.926598 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.926448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pssg9\" (UniqueName: \"kubernetes.io/projected/5adb8729-7a58-4cba-9f38-ea9ce848593e-kube-api-access-pssg9\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:11.928291 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.928230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5psz\" (UniqueName: \"kubernetes.io/projected/1972d46b-dd39-454c-b163-c2aa32794edf-kube-api-access-n5psz\") pod \"tuned-sqnst\" (UID: \"1972d46b-dd39-454c-b163-c2aa32794edf\") " pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:11.928824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:11.928801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw86x\" (UniqueName: \"kubernetes.io/projected/6809c636-8293-47c3-b326-7b09c2502d4c-kube-api-access-xw86x\") pod \"aws-ebs-csi-driver-node-pl467\" (UID: \"6809c636-8293-47c3-b326-7b09c2502d4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:12.008078 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.008003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:12.016311 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.016290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lgf6v"
Sep 09 03:13:12.024368 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.024347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdgpc"
Sep 09 03:13:12.029762 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.029732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zrhtb"
Sep 09 03:13:12.036467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.036446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rmm89"
Sep 09 03:13:12.043072 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.043054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467"
Sep 09 03:13:12.049816 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.049795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sqnst"
Sep 09 03:13:12.061504 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.061486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fv2sz"
Sep 09 03:13:12.064161 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.064144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:12.324245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.324172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:12.324245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.324243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:12.324432 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.324292    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:12.324432 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.324356    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:12.324432 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.324363    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:13.324343863 +0000 UTC m=+4.024107823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:12.324432 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.324371    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:12.324432 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.324384    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:12.324432 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.324420    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:13.324410099 +0000 UTC m=+4.024174060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:12.425368 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.425332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:12.425535 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.425502    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:12.425603 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.425565    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:13:13.425548445 +0000 UTC m=+4.125312386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:12.661161 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:12.661134    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27730a13_1bd3_4f06_a6cb_cd260ebc137b.slice/crio-91d4bfa2e2f989d2366e201ff4666e8b666dc2304959f540a1fc0ad261de6086 WatchSource:0}: Error finding container 91d4bfa2e2f989d2366e201ff4666e8b666dc2304959f540a1fc0ad261de6086: Status 404 returned error can't find the container with id 91d4bfa2e2f989d2366e201ff4666e8b666dc2304959f540a1fc0ad261de6086
Sep 09 03:13:12.662028 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:12.662002    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1972d46b_dd39_454c_b163_c2aa32794edf.slice/crio-5478083a478d03d7a820c20bbd5df4dee81208471570656e2bb901a75487c7a5 WatchSource:0}: Error finding container 5478083a478d03d7a820c20bbd5df4dee81208471570656e2bb901a75487c7a5: Status 404 returned error can't find the container with id 5478083a478d03d7a820c20bbd5df4dee81208471570656e2bb901a75487c7a5
Sep 09 03:13:12.662883 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:12.662781    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99922913_bda7_4f39_9dbe_ee67f87431f9.slice/crio-80776c24042240162f56219f6979fa4a0f94b9a4a4ecc16c623eee82632aa489 WatchSource:0}: Error finding container 80776c24042240162f56219f6979fa4a0f94b9a4a4ecc16c623eee82632aa489: Status 404 returned error can't find the container with id 80776c24042240162f56219f6979fa4a0f94b9a4a4ecc16c623eee82632aa489
Sep 09 03:13:12.663642 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:12.663623    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6809c636_8293_47c3_b326_7b09c2502d4c.slice/crio-ba7271a9489ba3bcf751c072a981f923755741e85de39f87f9f320db32cde3b0 WatchSource:0}: Error finding container ba7271a9489ba3bcf751c072a981f923755741e85de39f87f9f320db32cde3b0: Status 404 returned error can't find the container with id ba7271a9489ba3bcf751c072a981f923755741e85de39f87f9f320db32cde3b0
Sep 09 03:13:12.668754 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:12.668728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ed75e1_217e_49b7_939a_07720b22def2.slice/crio-9a25b5fbf85dfd4f9ea7c47a14ba823bc7e9e81939f27e7f47005ef6bd68545d WatchSource:0}: Error finding container 9a25b5fbf85dfd4f9ea7c47a14ba823bc7e9e81939f27e7f47005ef6bd68545d: Status 404 returned error can't find the container with id 9a25b5fbf85dfd4f9ea7c47a14ba823bc7e9e81939f27e7f47005ef6bd68545d
Sep 09 03:13:12.672981 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:12.672955    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca41c40_7763_4d20_9e3a_6fd665bccb52.slice/crio-44b8faaca21f300a0ef25328c4c1d81b4393c73ac60fdf423e180f16603a3c8b WatchSource:0}: Error finding container 44b8faaca21f300a0ef25328c4c1d81b4393c73ac60fdf423e180f16603a3c8b: Status 404 returned error can't find the container with id 44b8faaca21f300a0ef25328c4c1d81b4393c73ac60fdf423e180f16603a3c8b
Sep 09 03:13:12.750555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.750509    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 03:08:12 +0000 UTC" deadline="2028-06-04 08:16:00.088480828 +0000 UTC"
Sep 09 03:13:12.751166 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.750557    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15221h2m47.337928122s"
Sep 09 03:13:12.782498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.782478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:12.782595 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:12.782579    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:12.792464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.792437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdgpc" event={"ID":"1ca41c40-7763-4d20-9e3a-6fd665bccb52","Type":"ContainerStarted","Data":"44b8faaca21f300a0ef25328c4c1d81b4393c73ac60fdf423e180f16603a3c8b"}
Sep 09 03:13:12.795059 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.795034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jwkkl" event={"ID":"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b","Type":"ContainerStarted","Data":"e2a5a5cab002a87eead76e4f3e4099fd951b6018e4e2563d1e73c4549474d84e"}
Sep 09 03:13:12.796996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.796971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467" event={"ID":"6809c636-8293-47c3-b326-7b09c2502d4c","Type":"ContainerStarted","Data":"ba7271a9489ba3bcf751c072a981f923755741e85de39f87f9f320db32cde3b0"}
Sep 09 03:13:12.798292 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.798256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fv2sz" event={"ID":"27730a13-1bd3-4f06-a6cb-cd260ebc137b","Type":"ContainerStarted","Data":"91d4bfa2e2f989d2366e201ff4666e8b666dc2304959f540a1fc0ad261de6086"}
Sep 09 03:13:12.800207 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.800179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"534c9c617839cce2d3eb63410a25c41c072f81d12c06eead9385c4e2ea511b66"}
Sep 09 03:13:12.801550 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.801530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerStarted","Data":"9a25b5fbf85dfd4f9ea7c47a14ba823bc7e9e81939f27e7f47005ef6bd68545d"}
Sep 09 03:13:12.803671 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.803648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zrhtb" event={"ID":"99922913-bda7-4f39-9dbe-ee67f87431f9","Type":"ContainerStarted","Data":"80776c24042240162f56219f6979fa4a0f94b9a4a4ecc16c623eee82632aa489"}
Sep 09 03:13:12.805964 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.805916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sqnst" event={"ID":"1972d46b-dd39-454c-b163-c2aa32794edf","Type":"ContainerStarted","Data":"5478083a478d03d7a820c20bbd5df4dee81208471570656e2bb901a75487c7a5"}
Sep 09 03:13:12.807932 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.807898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal" event={"ID":"8b9765c33d70e82b31ab7040648baa6e","Type":"ContainerStarted","Data":"5954ca4d1d7f968fea395ada0fd9b07e62f80fb75919cd7fd895dc2b9ec6f6ab"}
Sep 09 03:13:12.809194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.809172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lgf6v" event={"ID":"db67cce9-62c6-45b2-b7af-24895bd4519b","Type":"ContainerStarted","Data":"c699d792462ea6253d68c2a822476fd0dd7e1b9110596ac6f49dc41a50ad2194"}
Sep 09 03:13:12.822776 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.822714    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-178.ec2.internal" podStartSLOduration=0.822701108 podStartE2EDuration="822.701108ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:12.822411249 +0000 UTC m=+3.522175214" watchObservedRunningTime="2026-09-09 03:13:12.822701108 +0000 UTC m=+3.522465074"
Sep 09 03:13:12.929791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:12.929703    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:13:13.332508 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.332372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:13.332508 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.332448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:13.332802 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.332625    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:13.332802 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.332644    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:13.332802 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.332657    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:13.333014 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.332813    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:13.333014 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.332898    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:15.332698426 +0000 UTC m=+6.032462370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:13.333014 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.332933    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:15.332920871 +0000 UTC m=+6.032684818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:13.433450 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.433397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:13.433630 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.433563    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:13.433630 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.433625    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:13:15.433606036 +0000 UTC m=+6.133369991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:13.752073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.751770    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 03:08:12 +0000 UTC" deadline="2028-04-21 15:15:19.852488027 +0000 UTC"
Sep 09 03:13:13.752073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.752039    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14172h2m6.100454907s"
Sep 09 03:13:13.786269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.783332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:13.786269 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.783454    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:13.786269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.786051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:13.786269 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:13.786174    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:13.844342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.843712    2580 generic.go:356] "Generic (PLEG): container finished" podID="bdb911bab22f2a07de1ddc19e29a1e21" containerID="eda2d20b8f0e6a93efe457bee9b1ecdb1cbcacef2911be32849b1879aa981790" exitCode=0
Sep 09 03:13:13.844342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:13.844241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal" event={"ID":"bdb911bab22f2a07de1ddc19e29a1e21","Type":"ContainerDied","Data":"eda2d20b8f0e6a93efe457bee9b1ecdb1cbcacef2911be32849b1879aa981790"}
Sep 09 03:13:14.782531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:14.782499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:14.782988 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:14.782643    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:14.858782 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:14.858003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal" event={"ID":"bdb911bab22f2a07de1ddc19e29a1e21","Type":"ContainerStarted","Data":"9857edc548199fd5ce37548486671fc00b5ca317043d0351b18e8c0847e63862"}
Sep 09 03:13:15.351248 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:15.351195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:15.351425 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:15.351264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:15.351425 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.351377    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:15.351425 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.351399    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:15.351425 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.351414    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:15.351425 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.351426    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:15.351703 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.351445    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:19.351426533 +0000 UTC m=+10.051190478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:15.351703 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.351463    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:19.351452244 +0000 UTC m=+10.051216188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:15.452744 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:15.452687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:15.452944 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.452899    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:15.453010 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.452960    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:13:19.452942036 +0000 UTC m=+10.152705981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:15.785451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:15.782941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:15.785451 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.783060    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:15.785451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:15.783760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:15.785451 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:15.783919    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:16.783734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:16.783522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:16.783734 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:16.783651    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:17.785606 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:17.783804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:17.785606 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:17.783947    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:17.785606 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:17.783809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:17.785606 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:17.784423    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:18.783087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:18.782989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:18.783265 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:18.783140    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:19.384798 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:19.384756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:19.384824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.384949    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.385036    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:27.385016682 +0000 UTC m=+18.084780623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.385035    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.385067    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.385082    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:19.385285 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.385135    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:27.385117333 +0000 UTC m=+18.084881283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:19.485955 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:19.485920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:19.486139 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.486074    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:19.486139 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.486132    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:13:27.486114115 +0000 UTC m=+18.185878071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:19.784561 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:19.783914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:19.784561 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.784024    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:19.784561 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:19.784366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:19.784561 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:19.784475    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:20.782626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:20.782588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:20.783147 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:20.782753    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:21.784493 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:21.784419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:21.784926 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:21.784536    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:21.784926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:21.784770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:21.784926 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:21.784890    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:22.782795 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:22.782763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:22.782957 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:22.782860    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:23.783511 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:23.783484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:23.783511 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:23.783503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:23.783940 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:23.783571    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:23.783940 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:23.783701    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:24.782896 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:24.782818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:24.783043 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:24.782923    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:25.783399 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:25.783366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:25.783835 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:25.783489    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:25.783835 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:25.783582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:25.783835 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:25.783725    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:26.783161 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:26.783114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:26.783400 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:26.783231    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:27.442888 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:27.442841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:27.442888 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:27.442889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:27.443424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.443004    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:27.443424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.443012    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:27.443424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.443078    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:43.443060729 +0000 UTC m=+34.142824682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:27.443424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.443020    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:27.443424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.443100    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:27.443424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.443123    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:13:43.443116804 +0000 UTC m=+34.142880744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:27.544164 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:27.544134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:27.544323 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.544308    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:27.544385 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.544375    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:13:43.544354038 +0000 UTC m=+34.244117978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:27.783309 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:27.783241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:27.783460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:27.783246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:27.783460 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.783372    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:27.783570 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:27.783472    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:28.783430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:28.783012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:28.783430 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:28.783125    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:29.783543 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:29.783509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:29.784009 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:29.783624    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:29.784009 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:29.783661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:29.784009 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:29.783849    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:30.783272 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:30.783236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:30.783440 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:30.783355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:31.782873 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.782851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:31.783561 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:31.782961    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:31.783561 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.783060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:31.783561 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:31.783177    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:31.895151 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.895131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zrhtb" event={"ID":"99922913-bda7-4f39-9dbe-ee67f87431f9","Type":"ContainerStarted","Data":"4085c980cb472f35e61ddd49491fa0ef3acb2639fa4a542fd9efe2e94527bf83"}
Sep 09 03:13:31.896641 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.896616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sqnst" event={"ID":"1972d46b-dd39-454c-b163-c2aa32794edf","Type":"ContainerStarted","Data":"cfec6a790b8433a8c56874dc1d3e63b76dc0bc24fa43840d6241c4d9ff5d84e2"}
Sep 09 03:13:31.898907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.898861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lgf6v" event={"ID":"db67cce9-62c6-45b2-b7af-24895bd4519b","Type":"ContainerStarted","Data":"e07d3b70918a6e6c999930946c90dfee27eaf8b79d533b00eb4ea0727a82a39e"}
Sep 09 03:13:31.900440 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.900417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdgpc" event={"ID":"1ca41c40-7763-4d20-9e3a-6fd665bccb52","Type":"ContainerStarted","Data":"c2fc328949c293d65233adf5fc21e65a61008de6176b0e91b0ebac2d1d847e81"}
Sep 09 03:13:31.904215 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.904189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jwkkl" event={"ID":"5bad8da5-211c-4a2a-adc6-130d5a3e7d9b","Type":"ContainerStarted","Data":"8120efea07d570648f56b29bfab9982b35b51de26bf7b28425248cd6e41f11e2"}
Sep 09 03:13:31.905582 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.905565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467" event={"ID":"6809c636-8293-47c3-b326-7b09c2502d4c","Type":"ContainerStarted","Data":"b7ba3f2450c6eae4c5c1081ab73e2cf1801313b8f9a34c24e5032d2e2b0f8386"}
Sep 09 03:13:31.908031 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.908007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"900fc2fd1484143af7f36b8b9e35f9ffd384ab9ba1e987912ece067e9a1c3a93"}
Sep 09 03:13:31.908265 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.908244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"a2c8c0037f00f852db8390e57ac2b984ee558b88942a4942825e61e07325f7b4"}
Sep 09 03:13:31.908324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.908302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"04c6c62dafdc25782e06d950e97255169bf3956d7bba7e39126c53c4532cb56a"}
Sep 09 03:13:31.908368 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.908352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"78694cc6a771b1c3ce863aa6aceb0a4aa4b48cf532e8def28dae996e4772e28d"}
Sep 09 03:13:31.909461 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.909442    2580 generic.go:356] "Generic (PLEG): container finished" podID="96ed75e1-217e-49b7-939a-07720b22def2" containerID="47865a2fdf7979709db70e2c12527c1afef6ab40da1293768ef025650567dc5b" exitCode=0
Sep 09 03:13:31.909556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.909471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerDied","Data":"47865a2fdf7979709db70e2c12527c1afef6ab40da1293768ef025650567dc5b"}
Sep 09 03:13:31.909916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.909876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zrhtb" podStartSLOduration=3.2128316630000002 podStartE2EDuration="21.909860875s" totalImagesPullingTime="18.697029212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:31.909861591 +0000 UTC m=+22.609625553" watchObservedRunningTime="2026-09-09 03:13:31.909860875 +0000 UTC m=+22.609624836"
Sep 09 03:13:31.910196 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.910161    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-178.ec2.internal" podStartSLOduration=19.910151275 podStartE2EDuration="19.910151275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:14.879214538 +0000 UTC m=+5.578978505" watchObservedRunningTime="2026-09-09 03:13:31.910151275 +0000 UTC m=+22.609915233"
Sep 09 03:13:31.968520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.968487    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kdgpc" podStartSLOduration=3.27675088 podStartE2EDuration="21.968476436s" totalImagesPullingTime="18.691725556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:31.96826769 +0000 UTC m=+22.668031652" watchObservedRunningTime="2026-09-09 03:13:31.968476436 +0000 UTC m=+22.668240397"
Sep 09 03:13:31.989912 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:31.989883    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lgf6v" podStartSLOduration=3.234395246 podStartE2EDuration="21.989870946s" totalImagesPullingTime="18.7554757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:31.989596622 +0000 UTC m=+22.689360586" watchObservedRunningTime="2026-09-09 03:13:31.989870946 +0000 UTC m=+22.689634921"
Sep 09 03:13:32.009427 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.009392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sqnst" podStartSLOduration=3.291785404 podStartE2EDuration="22.009381064s" totalImagesPullingTime="18.71759566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:32.009239084 +0000 UTC m=+22.709003047" watchObservedRunningTime="2026-09-09 03:13:32.009381064 +0000 UTC m=+22.709145026"
Sep 09 03:13:32.022884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.022853    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jwkkl" podStartSLOduration=3.350114032 podStartE2EDuration="22.022841545s" totalImagesPullingTime="18.672727513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:32.022521951 +0000 UTC m=+22.722285913" watchObservedRunningTime="2026-09-09 03:13:32.022841545 +0000 UTC m=+22.722605506"
Sep 09 03:13:32.783081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.782618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:32.783081 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:32.782911    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:32.863686 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.863646    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 03:13:32.913625 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.913585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467" event={"ID":"6809c636-8293-47c3-b326-7b09c2502d4c","Type":"ContainerStarted","Data":"0e9bf1445813b4b2d867dadaf887609fc9556f9a1dd84c78f211a9b24a24ea45"}
Sep 09 03:13:32.915306 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.915279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fv2sz" event={"ID":"27730a13-1bd3-4f06-a6cb-cd260ebc137b","Type":"ContainerStarted","Data":"8cdd08bf33861433d9d5599dcfdd9816f5cb2f50c3d1d8deec640a39dcf3c153"}
Sep 09 03:13:32.918925 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.918903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"d9cfc156af21364b1fc271022561cf6ae4a75c28d66028d8d417f527d3014766"}
Sep 09 03:13:32.919017 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.918979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"4495876dbc2d9b80830de0615479e8bcfe45a8fb9d34c11d42217c3f4b0ca41f"}
Sep 09 03:13:32.930621 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:32.930584    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fv2sz" podStartSLOduration=4.223744991 podStartE2EDuration="22.930571558s" totalImagesPullingTime="18.706826567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:32.930204873 +0000 UTC m=+23.629968834" watchObservedRunningTime="2026-09-09 03:13:32.930571558 +0000 UTC m=+23.630335519"
Sep 09 03:13:33.779607 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:33.779497    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T03:13:32.863663556Z","UUID":"cefe2c96-a9b9-4cfe-aeb2-b8655ed03ce7","Handler":null,"Name":"","Endpoint":""}
Sep 09 03:13:33.781265 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:33.781245    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 03:13:33.781406 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:33.781277    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 03:13:33.782651 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:33.782535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:33.783249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:33.783159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:33.783249 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:33.783054    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:33.784694 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:33.784641    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:34.783545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:34.783513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:34.784122 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:34.783651    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:34.927991 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:34.927953    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"fa1d1eaf3b1073120ae5d50001d414542b99dc3a7413fb0213b51386404461b2"}
Sep 09 03:13:34.930338 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:34.930311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467" event={"ID":"6809c636-8293-47c3-b326-7b09c2502d4c","Type":"ContainerStarted","Data":"3831511e83ed33dc40b8d73d6842d7c4649d0a53ec87f6268b139033b66f809c"}
Sep 09 03:13:34.947407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:34.947358    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pl467" podStartSLOduration=4.115396856 podStartE2EDuration="24.947339811s" totalImagesPullingTime="20.831942955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:34.946249559 +0000 UTC m=+25.646013522" watchObservedRunningTime="2026-09-09 03:13:34.947339811 +0000 UTC m=+25.647103822"
Sep 09 03:13:35.783010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:35.782977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:35.783190 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:35.783098    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:35.783190 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:35.783149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:35.783293 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:35.783266    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:35.946442 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:35.946408    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:35.946979 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:35.946962    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:36.783353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.783183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:36.783522 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:36.783440    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:36.937918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.937883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" event={"ID":"20739dea-771d-479e-9846-0f88eba250b8","Type":"ContainerStarted","Data":"c379e781027840cc7cef89a3af38630c668701faa9692f8b554dfb5ac2f12f46"}
Sep 09 03:13:36.938713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.938159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:36.938713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.938231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:36.938713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.938281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:36.940169 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.940150    2580 generic.go:356] "Generic (PLEG): container finished" podID="96ed75e1-217e-49b7-939a-07720b22def2" containerID="8d3083d53e9e86e1311b3ec2bfa474c77b888389e5a7d9e69f929315981e03a6" exitCode=0
Sep 09 03:13:36.940260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.940235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerDied","Data":"8d3083d53e9e86e1311b3ec2bfa474c77b888389e5a7d9e69f929315981e03a6"}
Sep 09 03:13:36.940390 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.940362    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:36.940915 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.940897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jwkkl"
Sep 09 03:13:36.954189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.954170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:36.954572 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.954257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:13:36.965183 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:36.965147    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp" podStartSLOduration=8.212235712 podStartE2EDuration="26.965135324s" totalImagesPullingTime="18.752899612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:36.964499772 +0000 UTC m=+27.664263774" watchObservedRunningTime="2026-09-09 03:13:36.965135324 +0000 UTC m=+27.664899283"
Sep 09 03:13:37.783340 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:37.783311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:37.783515 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:37.783311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:37.783515 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:37.783449    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:37.783515 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:37.783471    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:38.199640 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.199474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n498h"]
Sep 09 03:13:38.200074 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.199798    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:38.200074 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:38.199919    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:38.202653 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.202628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fb6s8"]
Sep 09 03:13:38.202786 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.202758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:38.202884 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:38.202863    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:38.203371 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.203350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tx8l4"]
Sep 09 03:13:38.203467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.203454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:38.203572 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:38.203551    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:38.947766 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.947736    2580 generic.go:356] "Generic (PLEG): container finished" podID="96ed75e1-217e-49b7-939a-07720b22def2" containerID="65047b492bfced888802cd935a78b1df69acff2bc91d592275a726eb19940755" exitCode=0
Sep 09 03:13:38.947944 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:38.947821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerDied","Data":"65047b492bfced888802cd935a78b1df69acff2bc91d592275a726eb19940755"}
Sep 09 03:13:39.783823 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:39.783800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:39.784260 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:39.783877    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:39.784260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:39.783951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:39.784260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:39.784022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:39.784260 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:39.784023    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:39.784260 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:39.784129    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:40.955461 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:40.955424    2580 generic.go:356] "Generic (PLEG): container finished" podID="96ed75e1-217e-49b7-939a-07720b22def2" containerID="7133448aeeb9d00081745cb828f079240f3ee321fd696896b900ecac73bed291" exitCode=0
Sep 09 03:13:40.955860 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:40.955476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerDied","Data":"7133448aeeb9d00081745cb828f079240f3ee321fd696896b900ecac73bed291"}
Sep 09 03:13:41.783277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:41.783198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:41.783277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:41.783210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:41.783277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:41.783225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:41.783570 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:41.783328    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:41.783570 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:41.783476    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:41.783570 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:41.783558    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:43.469432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:43.469392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:43.469455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.469542    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.469607    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.469622    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret podName:2d9cbca6-98ae-4c00-b9b6-c6368d560ff7 nodeName:}" failed. No retries permitted until 2026-09-09 03:14:15.469601632 +0000 UTC m=+66.169365595 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret") pod "global-pull-secret-syncer-tx8l4" (UID: "2d9cbca6-98ae-4c00-b9b6-c6368d560ff7") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.469624    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.469642    2580 projected.go:196] Error preparing data for projected volume kube-api-access-xmhv7 for pod openshift-network-diagnostics/network-check-target-n498h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:43.469919 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.469710    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7 podName:5269e3ca-1483-42a8-8be7-d29f78535e18 nodeName:}" failed. No retries permitted until 2026-09-09 03:14:15.469694411 +0000 UTC m=+66.169458351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmhv7" (UniqueName: "kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7") pod "network-check-target-n498h" (UID: "5269e3ca-1483-42a8-8be7-d29f78535e18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:13:43.570417 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:43.570380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:43.570589 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.570561    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:43.570633 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.570619    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:14:15.57060345 +0000 UTC m=+66.270367390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:13:43.782970 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:43.782900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:43.783118 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:43.782900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:43.783118 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:43.782910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:43.783118 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.783026    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tx8l4" podUID="2d9cbca6-98ae-4c00-b9b6-c6368d560ff7"
Sep 09 03:13:43.783118 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.783100    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n498h" podUID="5269e3ca-1483-42a8-8be7-d29f78535e18"
Sep 09 03:13:43.783334 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:43.783196    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:13:45.164881 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.164710    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-178.ec2.internal" event="NodeReady"
Sep 09 03:13:45.165144 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.164985    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 03:13:45.207903 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.207880    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-52xfk"]
Sep 09 03:13:45.210465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.210447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.210828 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.210712    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gzpdr"]
Sep 09 03:13:45.212867 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.212842    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 03:13:45.212867 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.212854    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pxbsq\""
Sep 09 03:13:45.213005 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.212936    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 03:13:45.213393 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.213370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.215364 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.215350    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 03:13:45.215462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.215447    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 03:13:45.215525 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.215481    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 03:13:45.215724 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.215710    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5kdc9\""
Sep 09 03:13:45.219721 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.219703    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-52xfk"]
Sep 09 03:13:45.222767 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.222715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gzpdr"]
Sep 09 03:13:45.383667 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.383617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92e422a5-0d5d-4642-86bf-e9f267ce6625-cert\") pod \"ingress-canary-gzpdr\" (UID: \"92e422a5-0d5d-4642-86bf-e9f267ce6625\") " pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.383667 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.383647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.383667 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.383664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t469\" (UniqueName: \"kubernetes.io/projected/41de35a1-e1a6-41b7-8bfb-4a18889db59c-kube-api-access-5t469\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.383838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.383741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8bfl\" (UniqueName: \"kubernetes.io/projected/92e422a5-0d5d-4642-86bf-e9f267ce6625-kube-api-access-q8bfl\") pod \"ingress-canary-gzpdr\" (UID: \"92e422a5-0d5d-4642-86bf-e9f267ce6625\") " pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.383838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.383763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41de35a1-e1a6-41b7-8bfb-4a18889db59c-tmp-dir\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.383838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.383800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41de35a1-e1a6-41b7-8bfb-4a18889db59c-config-volume\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.484481 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92e422a5-0d5d-4642-86bf-e9f267ce6625-cert\") pod \"ingress-canary-gzpdr\" (UID: \"92e422a5-0d5d-4642-86bf-e9f267ce6625\") " pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.484541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.484541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t469\" (UniqueName: \"kubernetes.io/projected/41de35a1-e1a6-41b7-8bfb-4a18889db59c-kube-api-access-5t469\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.484541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8bfl\" (UniqueName: \"kubernetes.io/projected/92e422a5-0d5d-4642-86bf-e9f267ce6625-kube-api-access-q8bfl\") pod \"ingress-canary-gzpdr\" (UID: \"92e422a5-0d5d-4642-86bf-e9f267ce6625\") " pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.484634 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41de35a1-e1a6-41b7-8bfb-4a18889db59c-tmp-dir\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.484634 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41de35a1-e1a6-41b7-8bfb-4a18889db59c-config-volume\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.484634 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:45.484615    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:45.484783 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:45.484667    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:13:45.984650307 +0000 UTC m=+36.684414262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:45.484960 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.484941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41de35a1-e1a6-41b7-8bfb-4a18889db59c-tmp-dir\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.485165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.485131    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41de35a1-e1a6-41b7-8bfb-4a18889db59c-config-volume\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.488580 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.488560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92e422a5-0d5d-4642-86bf-e9f267ce6625-cert\") pod \"ingress-canary-gzpdr\" (UID: \"92e422a5-0d5d-4642-86bf-e9f267ce6625\") " pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.493022 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.493002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8bfl\" (UniqueName: \"kubernetes.io/projected/92e422a5-0d5d-4642-86bf-e9f267ce6625-kube-api-access-q8bfl\") pod \"ingress-canary-gzpdr\" (UID: \"92e422a5-0d5d-4642-86bf-e9f267ce6625\") " pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.493116 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.493101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t469\" (UniqueName: \"kubernetes.io/projected/41de35a1-e1a6-41b7-8bfb-4a18889db59c-kube-api-access-5t469\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.526276 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.526260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gzpdr"
Sep 09 03:13:45.783612 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.783586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:13:45.783788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.783586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:13:45.783788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.783733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:13:45.786717 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.786696    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 03:13:45.786834 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.786740    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 03:13:45.787961 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.787943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 03:13:45.788069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.787970    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g4cct\""
Sep 09 03:13:45.788069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.787979    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rkcrq\""
Sep 09 03:13:45.788069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.787971    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 03:13:45.800972 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.800941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gzpdr"]
Sep 09 03:13:45.804637 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:13:45.804604    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e422a5_0d5d_4642_86bf_e9f267ce6625.slice/crio-0e8a0a5c2ee3b0463612f5ffe497eeb5e446f218a586cdfc4e4348bc8be59a77 WatchSource:0}: Error finding container 0e8a0a5c2ee3b0463612f5ffe497eeb5e446f218a586cdfc4e4348bc8be59a77: Status 404 returned error can't find the container with id 0e8a0a5c2ee3b0463612f5ffe497eeb5e446f218a586cdfc4e4348bc8be59a77
Sep 09 03:13:45.969997 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.969969    2580 generic.go:356] "Generic (PLEG): container finished" podID="96ed75e1-217e-49b7-939a-07720b22def2" containerID="85f945a056b7aa6ec1e202217c6ddb2941c14bf21ca878468c7e5937ab62b933" exitCode=0
Sep 09 03:13:45.970115 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.970047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerDied","Data":"85f945a056b7aa6ec1e202217c6ddb2941c14bf21ca878468c7e5937ab62b933"}
Sep 09 03:13:45.971148 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.971133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gzpdr" event={"ID":"92e422a5-0d5d-4642-86bf-e9f267ce6625","Type":"ContainerStarted","Data":"0e8a0a5c2ee3b0463612f5ffe497eeb5e446f218a586cdfc4e4348bc8be59a77"}
Sep 09 03:13:45.987378 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:45.987355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:45.987509 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:45.987492    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:45.987567 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:45.987557    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:13:46.987540391 +0000 UTC m=+37.687304331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:46.977445 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:46.977402    2580 generic.go:356] "Generic (PLEG): container finished" podID="96ed75e1-217e-49b7-939a-07720b22def2" containerID="9e896162dac37ff036694d754da7ffe10a730c27d848c84b654140b09f460b16" exitCode=0
Sep 09 03:13:46.978238 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:46.977450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerDied","Data":"9e896162dac37ff036694d754da7ffe10a730c27d848c84b654140b09f460b16"}
Sep 09 03:13:46.995379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:46.995354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:46.995526 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:46.995508    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:46.995580 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:46.995572    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:13:48.995554062 +0000 UTC m=+39.695318001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:47.982990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:47.982810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rmm89" event={"ID":"96ed75e1-217e-49b7-939a-07720b22def2","Type":"ContainerStarted","Data":"4f3ca1b2ba8c4eb3340fb34e74ea4b716016ac27c7e2fa7c1fd1a3c41ced4979"}
Sep 09 03:13:48.002817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:48.002773    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rmm89" podStartSLOduration=8.944941713 podStartE2EDuration="38.002758198s" totalImagesPullingTime="29.057816485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:48.001415511 +0000 UTC m=+38.701179496" watchObservedRunningTime="2026-09-09 03:13:48.002758198 +0000 UTC m=+38.702522150"
Sep 09 03:13:48.986448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:48.986413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gzpdr" event={"ID":"92e422a5-0d5d-4642-86bf-e9f267ce6625","Type":"ContainerStarted","Data":"37ba5874743f239763695ba83790e68be27b37e327f9b803fd10944c21deeb50"}
Sep 09 03:13:49.002366 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:49.002308    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gzpdr" podStartSLOduration=1.6031555210000001 podStartE2EDuration="4.002294231s" totalImagesPullingTime="2.39913871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:13:49.001183737 +0000 UTC m=+39.700947725" watchObservedRunningTime="2026-09-09 03:13:49.002294231 +0000 UTC m=+39.702058190"
Sep 09 03:13:49.010834 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:49.010805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:49.010979 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:49.010950    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:49.011038 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:49.011031    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:13:53.011011634 +0000 UTC m=+43.710775581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:13:53.042307 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:13:53.042277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:13:53.042644 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:53.042383    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:13:53.042644 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:13:53.042433    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:14:01.042419935 +0000 UTC m=+51.742183875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:01.096137 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:01.096098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:14:01.096562 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:01.096239    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:14:01.096562 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:01.096297    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:14:17.096282053 +0000 UTC m=+67.796045993 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:08.964987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:08.964958    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5sjp"
Sep 09 03:14:15.485943 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.485905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:14:15.485943 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.485960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:14:15.488833 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.488815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 03:14:15.488895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.488817    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 03:14:15.499151 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.499130    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 03:14:15.500377 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.500361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2d9cbca6-98ae-4c00-b9b6-c6368d560ff7-original-pull-secret\") pod \"global-pull-secret-syncer-tx8l4\" (UID: \"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7\") " pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:14:15.501927 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.501916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tx8l4"
Sep 09 03:14:15.510324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.510304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmhv7\" (UniqueName: \"kubernetes.io/projected/5269e3ca-1483-42a8-8be7-d29f78535e18-kube-api-access-xmhv7\") pod \"network-check-target-n498h\" (UID: \"5269e3ca-1483-42a8-8be7-d29f78535e18\") " pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:14:15.587493 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.587141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:14:15.589540 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.589517    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 03:14:15.597555 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:15.597537    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 03:14:15.597641 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:15.597606    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:15:19.597580997 +0000 UTC m=+130.297344937 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : secret "metrics-daemon-secret" not found
Sep 09 03:14:15.795851 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.795794    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tx8l4"]
Sep 09 03:14:15.798636 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:14:15.798615    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9cbca6_98ae_4c00_b9b6_c6368d560ff7.slice/crio-1214f5478241780dc8fe634250cd127008f05864501d3629c96ec48db1f5589a WatchSource:0}: Error finding container 1214f5478241780dc8fe634250cd127008f05864501d3629c96ec48db1f5589a: Status 404 returned error can't find the container with id 1214f5478241780dc8fe634250cd127008f05864501d3629c96ec48db1f5589a
Sep 09 03:14:15.809019 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.809002    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g4cct\""
Sep 09 03:14:15.817801 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:15.817754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:14:16.047278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:16.047216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tx8l4" event={"ID":"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7","Type":"ContainerStarted","Data":"1214f5478241780dc8fe634250cd127008f05864501d3629c96ec48db1f5589a"}
Sep 09 03:14:16.111973 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:16.111947    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n498h"]
Sep 09 03:14:16.114449 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:14:16.114427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5269e3ca_1483_42a8_8be7_d29f78535e18.slice/crio-c6b24c207f75daef379bb331881e8b5f2e00f699e4b6309f2c1c8700c9f03aa5 WatchSource:0}: Error finding container c6b24c207f75daef379bb331881e8b5f2e00f699e4b6309f2c1c8700c9f03aa5: Status 404 returned error can't find the container with id c6b24c207f75daef379bb331881e8b5f2e00f699e4b6309f2c1c8700c9f03aa5
Sep 09 03:14:17.052377 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:17.052311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n498h" event={"ID":"5269e3ca-1483-42a8-8be7-d29f78535e18","Type":"ContainerStarted","Data":"c6b24c207f75daef379bb331881e8b5f2e00f699e4b6309f2c1c8700c9f03aa5"}
Sep 09 03:14:17.098431 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:17.097870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:14:17.098431 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:17.098029    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:14:17.098431 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:17.098087    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:14:49.098073556 +0000 UTC m=+99.797837495 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:21.064969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:21.064935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n498h" event={"ID":"5269e3ca-1483-42a8-8be7-d29f78535e18","Type":"ContainerStarted","Data":"f8182615036c1b19aad2f057974301be084f8aab4c4a6e0f67b2dce85d2230bd"}
Sep 09 03:14:21.065455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:21.065012    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:14:21.066431 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:21.066413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tx8l4" event={"ID":"2d9cbca6-98ae-4c00-b9b6-c6368d560ff7","Type":"ContainerStarted","Data":"bcdb2489b2ee9d0c3fff2a9441362e8dd76c7c105bfe9c4b2804d388305d47a7"}
Sep 09 03:14:21.079202 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:21.079160    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n498h" podStartSLOduration=66.916586669 podStartE2EDuration="1m11.079148272s" totalImagesPullingTime="4.162561603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:14:21.078730156 +0000 UTC m=+71.778494122" watchObservedRunningTime="2026-09-09 03:14:21.079148272 +0000 UTC m=+71.778912287"
Sep 09 03:14:21.092143 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:21.092097    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tx8l4" podStartSLOduration=66.610975061 podStartE2EDuration="1m11.0920835s" totalImagesPullingTime="4.481108439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:14:21.091653732 +0000 UTC m=+71.791417704" watchObservedRunningTime="2026-09-09 03:14:21.0920835 +0000 UTC m=+71.791847463"
Sep 09 03:14:49.197114 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:49.197082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:14:49.197505 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:49.197179    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:14:49.197505 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:14:49.197238    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:15:53.197224052 +0000 UTC m=+163.896987992 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:14:52.071286 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:14:52.071257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n498h"
Sep 09 03:15:00.989032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.989000    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"]
Sep 09 03:15:00.990400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.990384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:00.993131 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.993105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 03:15:00.993274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.993250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 03:15:00.994126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.994110    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-khfnm\""
Sep 09 03:15:00.994213 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.994129    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 03:15:00.994821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:00.994806    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 03:15:01.004043 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.004025    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"]
Sep 09 03:15:01.073009 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.072974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nbbq\" (UniqueName: \"kubernetes.io/projected/550e5454-dcb1-43ff-a058-a13d69ba4c92-kube-api-access-6nbbq\") pod \"managed-serviceaccount-addon-agent-7d998485fb-m45rr\" (UID: \"550e5454-dcb1-43ff-a058-a13d69ba4c92\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.073121 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.073054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/550e5454-dcb1-43ff-a058-a13d69ba4c92-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d998485fb-m45rr\" (UID: \"550e5454-dcb1-43ff-a058-a13d69ba4c92\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.173820 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.173798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nbbq\" (UniqueName: \"kubernetes.io/projected/550e5454-dcb1-43ff-a058-a13d69ba4c92-kube-api-access-6nbbq\") pod \"managed-serviceaccount-addon-agent-7d998485fb-m45rr\" (UID: \"550e5454-dcb1-43ff-a058-a13d69ba4c92\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.173908 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.173849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/550e5454-dcb1-43ff-a058-a13d69ba4c92-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d998485fb-m45rr\" (UID: \"550e5454-dcb1-43ff-a058-a13d69ba4c92\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.176310 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.176292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/550e5454-dcb1-43ff-a058-a13d69ba4c92-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d998485fb-m45rr\" (UID: \"550e5454-dcb1-43ff-a058-a13d69ba4c92\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.181658 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.181638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nbbq\" (UniqueName: \"kubernetes.io/projected/550e5454-dcb1-43ff-a058-a13d69ba4c92-kube-api-access-6nbbq\") pod \"managed-serviceaccount-addon-agent-7d998485fb-m45rr\" (UID: \"550e5454-dcb1-43ff-a058-a13d69ba4c92\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.308450 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.308384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"
Sep 09 03:15:01.571217 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:01.571145    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr"]
Sep 09 03:15:01.574361 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:15:01.574335    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod550e5454_dcb1_43ff_a058_a13d69ba4c92.slice/crio-eca5254eda6b3dc6a5c27e5ed810225e57662e599ef189409cdbea66af5d38e7 WatchSource:0}: Error finding container eca5254eda6b3dc6a5c27e5ed810225e57662e599ef189409cdbea66af5d38e7: Status 404 returned error can't find the container with id eca5254eda6b3dc6a5c27e5ed810225e57662e599ef189409cdbea66af5d38e7
Sep 09 03:15:02.162614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:02.162580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr" event={"ID":"550e5454-dcb1-43ff-a058-a13d69ba4c92","Type":"ContainerStarted","Data":"eca5254eda6b3dc6a5c27e5ed810225e57662e599ef189409cdbea66af5d38e7"}
Sep 09 03:15:06.174498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:06.174427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr" event={"ID":"550e5454-dcb1-43ff-a058-a13d69ba4c92","Type":"ContainerStarted","Data":"e6e3fba6b5c0bd3be1bec0bd36c5a00bbfb25711f32be4eaa52f9e4dcfbee0cf"}
Sep 09 03:15:06.189813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:06.189769    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr" podStartSLOduration=1.879000805 podStartE2EDuration="6.189755023s" totalImagesPullingTime="4.310754218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:15:06.188627007 +0000 UTC m=+116.888390971" watchObservedRunningTime="2026-09-09 03:15:06.189755023 +0000 UTC m=+116.889518984"
Sep 09 03:15:19.597893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:19.597850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:15:19.598368 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:15:19.597974    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 03:15:19.598368 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:15:19.598029    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:17:21.598015211 +0000 UTC m=+252.297779151 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : secret "metrics-daemon-secret" not found
Sep 09 03:15:48.221493 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:15:48.221458    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-52xfk" podUID="41de35a1-e1a6-41b7-8bfb-4a18889db59c"
Sep 09 03:15:48.269463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:48.269440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-52xfk"
Sep 09 03:15:48.795918 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:15:48.795882    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:15:53.209945 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:15:53.209914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:15:53.210413 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:15:53.210047    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:15:53.210413 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:15:53.210098    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls podName:41de35a1-e1a6-41b7-8bfb-4a18889db59c nodeName:}" failed. No retries permitted until 2026-09-09 03:17:55.210084426 +0000 UTC m=+285.909848366 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls") pod "dns-default-52xfk" (UID: "41de35a1-e1a6-41b7-8bfb-4a18889db59c") : secret "dns-default-metrics-tls" not found
Sep 09 03:16:02.783211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:16:02.783130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:17:06.450779 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:06.450699    2580 generic.go:356] "Generic (PLEG): container finished" podID="550e5454-dcb1-43ff-a058-a13d69ba4c92" containerID="e6e3fba6b5c0bd3be1bec0bd36c5a00bbfb25711f32be4eaa52f9e4dcfbee0cf" exitCode=255
Sep 09 03:17:06.450779 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:06.450753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr" event={"ID":"550e5454-dcb1-43ff-a058-a13d69ba4c92","Type":"ContainerDied","Data":"e6e3fba6b5c0bd3be1bec0bd36c5a00bbfb25711f32be4eaa52f9e4dcfbee0cf"}
Sep 09 03:17:06.456306 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:06.456285    2580 scope.go:117] "RemoveContainer" containerID="e6e3fba6b5c0bd3be1bec0bd36c5a00bbfb25711f32be4eaa52f9e4dcfbee0cf"
Sep 09 03:17:07.454548 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:07.454517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d998485fb-m45rr" event={"ID":"550e5454-dcb1-43ff-a058-a13d69ba4c92","Type":"ContainerStarted","Data":"5f638714ac6b304fb1a0fe76305a4e387cf7b56d923936036931b688374ce32e"}
Sep 09 03:17:21.663799 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:21.663760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:17:21.664278 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:21.663898    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 03:17:21.664278 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:21.663965    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs podName:5adb8729-7a58-4cba-9f38-ea9ce848593e nodeName:}" failed. No retries permitted until 2026-09-09 03:19:23.663949908 +0000 UTC m=+374.363713848 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs") pod "network-metrics-daemon-fb6s8" (UID: "5adb8729-7a58-4cba-9f38-ea9ce848593e") : secret "metrics-daemon-secret" not found
Sep 09 03:17:23.646399 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.646366    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6fc79f467b-mq5wm"]
Sep 09 03:17:23.649348 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.649328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.651763 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.651738    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qlcgr\""
Sep 09 03:17:23.652017 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.652000    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 03:17:23.652090 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.652038    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 03:17:23.652090 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.652000    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 03:17:23.652197 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.652109    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 03:17:23.652197 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.652128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 03:17:23.652700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.652688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 03:17:23.675229 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.675208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fc79f467b-mq5wm"]
Sep 09 03:17:23.677519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.677485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-stats-auth\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.677607 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.677530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.677607 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.677573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jk8t\" (UniqueName: \"kubernetes.io/projected/17db3411-56eb-4092-bd50-3ce1544e58e4-kube-api-access-5jk8t\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.677726 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.677627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-default-certificate\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.677726 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.677664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.748703 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.748634    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"]
Sep 09 03:17:23.751628 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.751607    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"]
Sep 09 03:17:23.751783 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.751762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:23.754073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.754056    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55f5846475-gxmg5"]
Sep 09 03:17:23.754184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.754170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.754557 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.754533    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:17:23.754825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.754809    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 03:17:23.754896 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.754849    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-82wh8\""
Sep 09 03:17:23.755073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.755059    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 03:17:23.756378 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.756352    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 09 03:17:23.756378 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.756362    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 09 03:17:23.756612 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.756595    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-t6qbv"]
Sep 09 03:17:23.756765 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.756722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.756765 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.756743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-89ms7\""
Sep 09 03:17:23.756887 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.756797    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 09 03:17:23.757294 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.757275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:17:23.759061 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.759039    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 03:17:23.759184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.759161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 03:17:23.759306 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.759167    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 03:17:23.759306 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.759283    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dmprq\""
Sep 09 03:17:23.759435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.759389    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv"
Sep 09 03:17:23.761978 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.761959    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7gn59\""
Sep 09 03:17:23.765576 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.765557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 03:17:23.766022 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.766000    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-t6qbv"]
Sep 09 03:17:23.767135 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.767117    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"]
Sep 09 03:17:23.768229 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.768210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"]
Sep 09 03:17:23.778381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-stats-auth\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.778464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tzww\" (UniqueName: \"kubernetes.io/projected/45278e8c-dfe9-4a77-a02a-03ac57df2774-kube-api-access-8tzww\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:23.778464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scrnp\" (UniqueName: \"kubernetes.io/projected/b4247b76-d174-41a9-a607-ecc4994781a5-kube-api-access-scrnp\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.778464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.778464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jk8t\" (UniqueName: \"kubernetes.io/projected/17db3411-56eb-4092-bd50-3ce1544e58e4-kube-api-access-5jk8t\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.778661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kcxz\" (UniqueName: \"kubernetes.io/projected/435efb36-78d8-498b-a9a2-ac80492ba57a-kube-api-access-2kcxz\") pod \"network-check-source-685456655f-t6qbv\" (UID: \"435efb36-78d8-498b-a9a2-ac80492ba57a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv"
Sep 09 03:17:23.778661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-default-certificate\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.778661 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.778592    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:24.278574291 +0000 UTC m=+254.978338230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : configmap references non-existent config key: service-ca.crt
Sep 09 03:17:23.778836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.778836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:23.778836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-image-registry-private-configuration\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.778836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-installation-pull-secrets\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.778836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4247b76-d174-41a9-a607-ecc4994781a5-config\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-bound-sa-token\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4247b76-d174-41a9-a607-ecc4994781a5-serving-cert\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl79z\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-kube-api-access-sl79z\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-trusted-ca\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.778977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0fd1dd10-737a-4a07-9087-50bc04d158d5-ca-trust-extracted\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.779007    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:17:23.779057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.779015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-certificates\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.779332 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.779216    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:24.279116931 +0000 UTC m=+254.978880884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : secret "router-metrics-certs-default" not found
Sep 09 03:17:23.780967 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.780947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-stats-auth\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.781353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.781306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-default-certificate\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.782470 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.782436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55f5846475-gxmg5"]
Sep 09 03:17:23.790480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.790460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jk8t\" (UniqueName: \"kubernetes.io/projected/17db3411-56eb-4092-bd50-3ce1544e58e4-kube-api-access-5jk8t\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:23.879638 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcxz\" (UniqueName: \"kubernetes.io/projected/435efb36-78d8-498b-a9a2-ac80492ba57a-kube-api-access-2kcxz\") pod \"network-check-source-685456655f-t6qbv\" (UID: \"435efb36-78d8-498b-a9a2-ac80492ba57a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv"
Sep 09 03:17:23.879739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.879739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:23.879739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-image-registry-private-configuration\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.879739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-installation-pull-secrets\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.879739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4247b76-d174-41a9-a607-ecc4994781a5-config\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-bound-sa-token\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.879795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4247b76-d174-41a9-a607-ecc4994781a5-serving-cert\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.879762    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.879842    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55f5846475-gxmg5: secret "image-registry-tls" not found
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.879904    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls podName:0fd1dd10-737a-4a07-9087-50bc04d158d5 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:24.379885873 +0000 UTC m=+255.079649816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls") pod "image-registry-55f5846475-gxmg5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5") : secret "image-registry-tls" not found
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.879772    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:17:23.879974 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:23.879964    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls podName:45278e8c-dfe9-4a77-a02a-03ac57df2774 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:24.379947869 +0000 UTC m=+255.079711809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-k5z5m" (UID: "45278e8c-dfe9-4a77-a02a-03ac57df2774") : secret "samples-operator-tls" not found
Sep 09 03:17:23.880324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl79z\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-kube-api-access-sl79z\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.880324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-trusted-ca\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.880324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0fd1dd10-737a-4a07-9087-50bc04d158d5-ca-trust-extracted\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.880324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-certificates\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.880324 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tzww\" (UniqueName: \"kubernetes.io/projected/45278e8c-dfe9-4a77-a02a-03ac57df2774-kube-api-access-8tzww\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:23.880767 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scrnp\" (UniqueName: \"kubernetes.io/projected/b4247b76-d174-41a9-a607-ecc4994781a5-kube-api-access-scrnp\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.880767 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4247b76-d174-41a9-a607-ecc4994781a5-config\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.880910 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.880884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0fd1dd10-737a-4a07-9087-50bc04d158d5-ca-trust-extracted\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.881307 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.881288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-trusted-ca\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.881466 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.881443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-certificates\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.882175 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.882149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-installation-pull-secrets\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.882249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.882182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-image-registry-private-configuration\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.882249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.882239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4247b76-d174-41a9-a607-ecc4994781a5-serving-cert\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.889318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.889289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-bound-sa-token\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.889519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.889501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl79z\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-kube-api-access-sl79z\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:23.889691 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.889650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kcxz\" (UniqueName: \"kubernetes.io/projected/435efb36-78d8-498b-a9a2-ac80492ba57a-kube-api-access-2kcxz\") pod \"network-check-source-685456655f-t6qbv\" (UID: \"435efb36-78d8-498b-a9a2-ac80492ba57a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv"
Sep 09 03:17:23.889921 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.889904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scrnp\" (UniqueName: \"kubernetes.io/projected/b4247b76-d174-41a9-a607-ecc4994781a5-kube-api-access-scrnp\") pod \"service-ca-operator-859f94c4b-x9czc\" (UID: \"b4247b76-d174-41a9-a607-ecc4994781a5\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:23.890136 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:23.890120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tzww\" (UniqueName: \"kubernetes.io/projected/45278e8c-dfe9-4a77-a02a-03ac57df2774-kube-api-access-8tzww\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:24.068258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.068237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"
Sep 09 03:17:24.079275 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.079251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv"
Sep 09 03:17:24.282709 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.282653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:24.282839 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.282759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:24.282839 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.282816    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:25.282797351 +0000 UTC m=+255.982561291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : configmap references non-existent config key: service-ca.crt
Sep 09 03:17:24.282968 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.282876    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:17:24.282968 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.282935    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:25.282917458 +0000 UTC m=+255.982681417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : secret "router-metrics-certs-default" not found
Sep 09 03:17:24.335264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.335210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc"]
Sep 09 03:17:24.338272 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:17:24.338251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4247b76_d174_41a9_a607_ecc4994781a5.slice/crio-a6a691ef3f1b033dd766ef7ae371b8d9b6f58254daaacbfc84c234a5e299f29f WatchSource:0}: Error finding container a6a691ef3f1b033dd766ef7ae371b8d9b6f58254daaacbfc84c234a5e299f29f: Status 404 returned error can't find the container with id a6a691ef3f1b033dd766ef7ae371b8d9b6f58254daaacbfc84c234a5e299f29f
Sep 09 03:17:24.355362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.355343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-t6qbv"]
Sep 09 03:17:24.357728 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:17:24.357709    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435efb36_78d8_498b_a9a2_ac80492ba57a.slice/crio-64a74c420edcafa4f26a7a5f02c325d6874d57af9167233d9c1f32f92d9e515a WatchSource:0}: Error finding container 64a74c420edcafa4f26a7a5f02c325d6874d57af9167233d9c1f32f92d9e515a: Status 404 returned error can't find the container with id 64a74c420edcafa4f26a7a5f02c325d6874d57af9167233d9c1f32f92d9e515a
Sep 09 03:17:24.384060 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.384032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:24.384146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.384072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:24.384185 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.384171    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:17:24.384218 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.384186    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55f5846475-gxmg5: secret "image-registry-tls" not found
Sep 09 03:17:24.384250 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.384230    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls podName:0fd1dd10-737a-4a07-9087-50bc04d158d5 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:25.38421601 +0000 UTC m=+256.083979950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls") pod "image-registry-55f5846475-gxmg5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5") : secret "image-registry-tls" not found
Sep 09 03:17:24.384250 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.384231    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:17:24.384315 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:24.384273    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls podName:45278e8c-dfe9-4a77-a02a-03ac57df2774 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:25.384261113 +0000 UTC m=+256.084025054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-k5z5m" (UID: "45278e8c-dfe9-4a77-a02a-03ac57df2774") : secret "samples-operator-tls" not found
Sep 09 03:17:24.501216 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.501179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv" event={"ID":"435efb36-78d8-498b-a9a2-ac80492ba57a","Type":"ContainerStarted","Data":"d426ae4a7ef6ff117a144271b1ad9bf3288e69ff1672f5a07320abad3886a41e"}
Sep 09 03:17:24.501360 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.501271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv" event={"ID":"435efb36-78d8-498b-a9a2-ac80492ba57a","Type":"ContainerStarted","Data":"64a74c420edcafa4f26a7a5f02c325d6874d57af9167233d9c1f32f92d9e515a"}
Sep 09 03:17:24.502369 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.502347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc" event={"ID":"b4247b76-d174-41a9-a607-ecc4994781a5","Type":"ContainerStarted","Data":"a6a691ef3f1b033dd766ef7ae371b8d9b6f58254daaacbfc84c234a5e299f29f"}
Sep 09 03:17:24.515474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:24.515426    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-t6qbv" podStartSLOduration=1.515411296 podStartE2EDuration="1.515411296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:24.515092277 +0000 UTC m=+255.214856238" watchObservedRunningTime="2026-09-09 03:17:24.515411296 +0000 UTC m=+255.215175261"
Sep 09 03:17:25.291337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:25.291301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:25.291728 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:25.291361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:25.291728 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.291469    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:27.291450546 +0000 UTC m=+257.991214485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : configmap references non-existent config key: service-ca.crt
Sep 09 03:17:25.291728 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.291515    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:17:25.291728 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.291563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:27.291551668 +0000 UTC m=+257.991315608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : secret "router-metrics-certs-default" not found
Sep 09 03:17:25.392244 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:25.392209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:25.392244 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:25.392244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:25.392456 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.392341    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:17:25.392456 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.392355    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:17:25.392456 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.392376    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55f5846475-gxmg5: secret "image-registry-tls" not found
Sep 09 03:17:25.392456 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.392387    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls podName:45278e8c-dfe9-4a77-a02a-03ac57df2774 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:27.392375512 +0000 UTC m=+258.092139451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-k5z5m" (UID: "45278e8c-dfe9-4a77-a02a-03ac57df2774") : secret "samples-operator-tls" not found
Sep 09 03:17:25.392456 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:25.392417    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls podName:0fd1dd10-737a-4a07-9087-50bc04d158d5 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:27.392403803 +0000 UTC m=+258.092167742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls") pod "image-registry-55f5846475-gxmg5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5") : secret "image-registry-tls" not found
Sep 09 03:17:26.510301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:26.510268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc" event={"ID":"b4247b76-d174-41a9-a607-ecc4994781a5","Type":"ContainerStarted","Data":"5667e49b28a3fdd970ff2b172c4218907f70d1541a1b55f4746026d62c5830d6"}
Sep 09 03:17:26.526124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:26.526072    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc" podStartSLOduration=1.505318398 podStartE2EDuration="3.526058013s" totalImagesPullingTime="2.020739615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:26.525111803 +0000 UTC m=+257.224875776" watchObservedRunningTime="2026-09-09 03:17:26.526058013 +0000 UTC m=+257.225821975"
Sep 09 03:17:27.307180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:27.307144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:27.307424 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:27.307197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:27.307424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.307289    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:17:27.307424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.307303    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:31.307286145 +0000 UTC m=+262.007050085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : configmap references non-existent config key: service-ca.crt
Sep 09 03:17:27.307424 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.307327    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:31.307316862 +0000 UTC m=+262.007080801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : secret "router-metrics-certs-default" not found
Sep 09 03:17:27.407743 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:27.407698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:27.407743 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:27.407745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:27.407936 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.407835    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:17:27.407936 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.407840    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:17:27.407936 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.407858    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55f5846475-gxmg5: secret "image-registry-tls" not found
Sep 09 03:17:27.407936 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.407883    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls podName:45278e8c-dfe9-4a77-a02a-03ac57df2774 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:31.407870387 +0000 UTC m=+262.107634331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-k5z5m" (UID: "45278e8c-dfe9-4a77-a02a-03ac57df2774") : secret "samples-operator-tls" not found
Sep 09 03:17:27.407936 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:27.407897    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls podName:0fd1dd10-737a-4a07-9087-50bc04d158d5 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:31.4078911 +0000 UTC m=+262.107655041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls") pod "image-registry-55f5846475-gxmg5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5") : secret "image-registry-tls" not found
Sep 09 03:17:29.097265 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.097234    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nm89b"]
Sep 09 03:17:29.100448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.100433    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.102742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.102718    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 03:17:29.102742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.102729    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h5g47\""
Sep 09 03:17:29.103646 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.103626    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 03:17:29.103646 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.103629    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 03:17:29.103831 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.103686    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 03:17:29.109511 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.109490    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nm89b"]
Sep 09 03:17:29.119829 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.119804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.119926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.119834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11dde6d7-611d-4a45-99a7-c30ad55f00ca-data-volume\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.119926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.119864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11dde6d7-611d-4a45-99a7-c30ad55f00ca-crio-socket\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.119998 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.119972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.119998 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.119992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlxvq\" (UniqueName: \"kubernetes.io/projected/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-api-access-nlxvq\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.221597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.221643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11dde6d7-611d-4a45-99a7-c30ad55f00ca-data-volume\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.221708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11dde6d7-611d-4a45-99a7-c30ad55f00ca-crio-socket\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.221799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.221829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxvq\" (UniqueName: \"kubernetes.io/projected/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-api-access-nlxvq\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.222252    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.222318    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:29.722299489 +0000 UTC m=+260.422063435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.222783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11dde6d7-611d-4a45-99a7-c30ad55f00ca-data-volume\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.222870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11dde6d7-611d-4a45-99a7-c30ad55f00ca-crio-socket\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.222936    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.224741 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.222982    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:29.722967916 +0000 UTC m=+260.422731869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.232370 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.232349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxvq\" (UniqueName: \"kubernetes.io/projected/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-api-access-nlxvq\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.726463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.726437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.726626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.726501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:29.726626 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.726565    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.726626 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.726593    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.726765 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.726645    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:30.726626303 +0000 UTC m=+261.426390248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:29.726765 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:29.726662    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:30.726651366 +0000 UTC m=+261.426415328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:29.777685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.777648    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"]
Sep 09 03:17:29.780821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.780807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"
Sep 09 03:17:29.785808 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.785788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4slj4\""
Sep 09 03:17:29.786772 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.786749    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 03:17:29.786772 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.786762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 03:17:29.794327 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.794309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"]
Sep 09 03:17:29.827494 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.827462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdc7k\" (UniqueName: \"kubernetes.io/projected/30a07227-9188-4ada-8a0d-87ccbb6e728e-kube-api-access-fdc7k\") pod \"migrator-fdbb8fdff-nvjg5\" (UID: \"30a07227-9188-4ada-8a0d-87ccbb6e728e\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"
Sep 09 03:17:29.928021 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.927995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdc7k\" (UniqueName: \"kubernetes.io/projected/30a07227-9188-4ada-8a0d-87ccbb6e728e-kube-api-access-fdc7k\") pod \"migrator-fdbb8fdff-nvjg5\" (UID: \"30a07227-9188-4ada-8a0d-87ccbb6e728e\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"
Sep 09 03:17:29.937590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:29.937566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdc7k\" (UniqueName: \"kubernetes.io/projected/30a07227-9188-4ada-8a0d-87ccbb6e728e-kube-api-access-fdc7k\") pod \"migrator-fdbb8fdff-nvjg5\" (UID: \"30a07227-9188-4ada-8a0d-87ccbb6e728e\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"
Sep 09 03:17:30.090446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:30.090394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"
Sep 09 03:17:30.360238 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:30.360178    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5"]
Sep 09 03:17:30.362995 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:17:30.362971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a07227_9188_4ada_8a0d_87ccbb6e728e.slice/crio-5ab6d1cc525f9dddab06623655284a01e1f64b7dd6e6b689c693fc92eee09e6e WatchSource:0}: Error finding container 5ab6d1cc525f9dddab06623655284a01e1f64b7dd6e6b689c693fc92eee09e6e: Status 404 returned error can't find the container with id 5ab6d1cc525f9dddab06623655284a01e1f64b7dd6e6b689c693fc92eee09e6e
Sep 09 03:17:30.522844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:30.522819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5" event={"ID":"30a07227-9188-4ada-8a0d-87ccbb6e728e","Type":"ContainerStarted","Data":"5ab6d1cc525f9dddab06623655284a01e1f64b7dd6e6b689c693fc92eee09e6e"}
Sep 09 03:17:30.735510 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:30.735481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:30.735667 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:30.735564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:30.735667 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:30.735604    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:30.735667 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:30.735666    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:32.735649985 +0000 UTC m=+263.435413929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:30.735880 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:30.735700    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:30.735880 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:30.735746    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:32.735734176 +0000 UTC m=+263.435498115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:31.059516 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:31.059432    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdgpc_1ca41c40-7763-4d20-9e3a-6fd665bccb52/dns-node-resolver/0.log"
Sep 09 03:17:31.340379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:31.340310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:31.340379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:31.340372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:31.340549 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.340457    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:17:31.340549 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.340511    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:39.340494887 +0000 UTC m=+270.040258827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : configmap references non-existent config key: service-ca.crt
Sep 09 03:17:31.340549 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.340524    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:39.34051885 +0000 UTC m=+270.040282789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : secret "router-metrics-certs-default" not found
Sep 09 03:17:31.441501 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:31.441471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:31.441929 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:31.441559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:31.441929 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.441613    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:17:31.441929 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.441631    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55f5846475-gxmg5: secret "image-registry-tls" not found
Sep 09 03:17:31.441929 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.441711    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:17:31.441929 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.441703    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls podName:0fd1dd10-737a-4a07-9087-50bc04d158d5 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:39.441685602 +0000 UTC m=+270.141449555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls") pod "image-registry-55f5846475-gxmg5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5") : secret "image-registry-tls" not found
Sep 09 03:17:31.441929 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:31.441796    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls podName:45278e8c-dfe9-4a77-a02a-03ac57df2774 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:39.441779072 +0000 UTC m=+270.141543018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-k5z5m" (UID: "45278e8c-dfe9-4a77-a02a-03ac57df2774") : secret "samples-operator-tls" not found
Sep 09 03:17:32.260264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.260239    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zrhtb_99922913-bda7-4f39-9dbe-ee67f87431f9/node-ca/0.log"
Sep 09 03:17:32.531020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.530959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5" event={"ID":"30a07227-9188-4ada-8a0d-87ccbb6e728e","Type":"ContainerStarted","Data":"bc9c33fceabfb349099e15a1dbb5b745764cf3d2e7d85dd5d4c0d3c671859233"}
Sep 09 03:17:32.531020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.531003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5" event={"ID":"30a07227-9188-4ada-8a0d-87ccbb6e728e","Type":"ContainerStarted","Data":"e0fc0ef6215ccde2f459bedb3309bbee1edbbdf48581f6bb70048a1996486df9"}
Sep 09 03:17:32.548289 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.548249    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-nvjg5" podStartSLOduration=2.22338483 podStartE2EDuration="3.548235324s" totalImagesPullingTime="1.324850494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:32.546762943 +0000 UTC m=+263.246526899" watchObservedRunningTime="2026-09-09 03:17:32.548235324 +0000 UTC m=+263.247999285"
Sep 09 03:17:32.752656 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.752629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:32.752818 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.752718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:32.752818 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:32.752763    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:32.752818 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:32.752799    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:32.752928 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:32.752822    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:36.752808059 +0000 UTC m=+267.452571998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:32.752928 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:32.752838    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:36.752828297 +0000 UTC m=+267.452592237 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:32.859416 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:32.859360    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gzpdr_92e422a5-0d5d-4642-86bf-e9f267ce6625/serve-healthcheck-canary/0.log"
Sep 09 03:17:36.782708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:36.782661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:36.783104 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:36.782743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:36.783104 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:36.782808    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:17:36.783104 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:36.782808    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:36.783104 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:36.782855    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:44.782843439 +0000 UTC m=+275.482607379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : configmap "kube-rbac-proxy" not found
Sep 09 03:17:36.783104 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:36.782873    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls podName:11dde6d7-611d-4a45-99a7-c30ad55f00ca nodeName:}" failed. No retries permitted until 2026-09-09 03:17:44.782860262 +0000 UTC m=+275.482624202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nm89b" (UID: "11dde6d7-611d-4a45-99a7-c30ad55f00ca") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:17:39.405009 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.404979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:39.405390 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.405034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:39.405390 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:39.405143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle podName:17db3411-56eb-4092-bd50-3ce1544e58e4 nodeName:}" failed. No retries permitted until 2026-09-09 03:17:55.405130362 +0000 UTC m=+286.104894301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle") pod "router-default-6fc79f467b-mq5wm" (UID: "17db3411-56eb-4092-bd50-3ce1544e58e4") : configmap references non-existent config key: service-ca.crt
Sep 09 03:17:39.407410 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.407390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17db3411-56eb-4092-bd50-3ce1544e58e4-metrics-certs\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:39.505288 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.505260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:39.505440 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.505334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:39.507722 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.507696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"image-registry-55f5846475-gxmg5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") " pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:39.507817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.507749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/45278e8c-dfe9-4a77-a02a-03ac57df2774-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-k5z5m\" (UID: \"45278e8c-dfe9-4a77-a02a-03ac57df2774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:39.660695 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.660608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"
Sep 09 03:17:39.674553 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.674531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:39.928409 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.928340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m"]
Sep 09 03:17:39.956990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:39.954438    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55f5846475-gxmg5"]
Sep 09 03:17:40.555692 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:40.555643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" event={"ID":"0fd1dd10-737a-4a07-9087-50bc04d158d5","Type":"ContainerStarted","Data":"aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5"}
Sep 09 03:17:40.556084 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:40.555712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" event={"ID":"0fd1dd10-737a-4a07-9087-50bc04d158d5","Type":"ContainerStarted","Data":"bd870901f131491b0566bdd5ebe625821381e9527ee0a151e1fd0f8933eb4243"}
Sep 09 03:17:40.556084 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:40.555743    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:17:40.556842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:40.556820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m" event={"ID":"45278e8c-dfe9-4a77-a02a-03ac57df2774","Type":"ContainerStarted","Data":"7ef45c7ea3adf22ef2b77d31714513ae80b713fb368ce53f52e076ce9a56716c"}
Sep 09 03:17:40.576030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:40.575986    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" podStartSLOduration=17.575973116 podStartE2EDuration="17.575973116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:40.574694731 +0000 UTC m=+271.274458694" watchObservedRunningTime="2026-09-09 03:17:40.575973116 +0000 UTC m=+271.275737110"
Sep 09 03:17:42.566095 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:42.566060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m" event={"ID":"45278e8c-dfe9-4a77-a02a-03ac57df2774","Type":"ContainerStarted","Data":"ed2598fc4783d6448021ffa6badec6b2fb82e41fd5b87e0313772b641ef20f50"}
Sep 09 03:17:42.566459 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:42.566113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m" event={"ID":"45278e8c-dfe9-4a77-a02a-03ac57df2774","Type":"ContainerStarted","Data":"014c73c781f612381009944d4cdc7b2d1e52380ff24c1dfdd34dc7351626a5da"}
Sep 09 03:17:42.582584 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:42.582538    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-k5z5m" podStartSLOduration=17.190649142 podStartE2EDuration="19.582520181s" totalImagesPullingTime="2.391871039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:42.581665132 +0000 UTC m=+273.281429095" watchObservedRunningTime="2026-09-09 03:17:42.582520181 +0000 UTC m=+273.282284164"
Sep 09 03:17:44.845805 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:44.845776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:44.846194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:44.845836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:44.846270 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:44.846251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11dde6d7-611d-4a45-99a7-c30ad55f00ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:44.848173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:44.848156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11dde6d7-611d-4a45-99a7-c30ad55f00ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nm89b\" (UID: \"11dde6d7-611d-4a45-99a7-c30ad55f00ca\") " pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:45.009842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:45.009806    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nm89b"
Sep 09 03:17:45.275819 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:45.275791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nm89b"]
Sep 09 03:17:45.278510 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:17:45.278491    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dde6d7_611d_4a45_99a7_c30ad55f00ca.slice/crio-3b5d1c5b767d0ed769770d4d653258b3acc7c72243590b08ae72fa807f83d7ae WatchSource:0}: Error finding container 3b5d1c5b767d0ed769770d4d653258b3acc7c72243590b08ae72fa807f83d7ae: Status 404 returned error can't find the container with id 3b5d1c5b767d0ed769770d4d653258b3acc7c72243590b08ae72fa807f83d7ae
Sep 09 03:17:45.576581 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:45.576512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nm89b" event={"ID":"11dde6d7-611d-4a45-99a7-c30ad55f00ca","Type":"ContainerStarted","Data":"9618b59f39fb4251dbc0461664334039a24b97e680a52fbf263450f37429e5b9"}
Sep 09 03:17:45.576581 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:45.576564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nm89b" event={"ID":"11dde6d7-611d-4a45-99a7-c30ad55f00ca","Type":"ContainerStarted","Data":"3b5d1c5b767d0ed769770d4d653258b3acc7c72243590b08ae72fa807f83d7ae"}
Sep 09 03:17:46.583296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:46.583267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nm89b" event={"ID":"11dde6d7-611d-4a45-99a7-c30ad55f00ca","Type":"ContainerStarted","Data":"e090aa55879f8bd3e1ba3f6c846394bab87252e0ec0a7c58da80749e052c3cae"}
Sep 09 03:17:48.592753 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:48.592720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nm89b" event={"ID":"11dde6d7-611d-4a45-99a7-c30ad55f00ca","Type":"ContainerStarted","Data":"b1bd4911c57778b1dd2ca2b2400c663f014d2fc64c2511060a49508269f846e6"}
Sep 09 03:17:48.610099 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:48.610053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nm89b" podStartSLOduration=17.2966639 podStartE2EDuration="19.61004018s" totalImagesPullingTime="2.31337628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:17:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:48.609733418 +0000 UTC m=+279.309497380" watchObservedRunningTime="2026-09-09 03:17:48.61004018 +0000 UTC m=+279.309804142"
Sep 09 03:17:51.270117 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:17:51.270078    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-52xfk" podUID="41de35a1-e1a6-41b7-8bfb-4a18889db59c"
Sep 09 03:17:51.602801 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:51.602727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-52xfk"
Sep 09 03:17:51.978255 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:51.978225    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55f5846475-gxmg5"]
Sep 09 03:17:55.212798 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.212766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:17:55.215198 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.215176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41de35a1-e1a6-41b7-8bfb-4a18889db59c-metrics-tls\") pod \"dns-default-52xfk\" (UID: \"41de35a1-e1a6-41b7-8bfb-4a18889db59c\") " pod="openshift-dns/dns-default-52xfk"
Sep 09 03:17:55.414254 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.414220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:55.414719 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.414704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17db3411-56eb-4092-bd50-3ce1544e58e4-service-ca-bundle\") pod \"router-default-6fc79f467b-mq5wm\" (UID: \"17db3411-56eb-4092-bd50-3ce1544e58e4\") " pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:55.458296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.458278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:55.506278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.506040    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pxbsq\""
Sep 09 03:17:55.514328 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.514131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-52xfk"
Sep 09 03:17:55.728071 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.728018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fc79f467b-mq5wm"]
Sep 09 03:17:55.729992 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:17:55.729967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17db3411_56eb_4092_bd50_3ce1544e58e4.slice/crio-f738880f735b8c2dfccaed951cb715790dfef2c17578470f8cd2dc9cdfe4e823 WatchSource:0}: Error finding container f738880f735b8c2dfccaed951cb715790dfef2c17578470f8cd2dc9cdfe4e823: Status 404 returned error can't find the container with id f738880f735b8c2dfccaed951cb715790dfef2c17578470f8cd2dc9cdfe4e823
Sep 09 03:17:55.791226 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:17:55.791200    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41de35a1_e1a6_41b7_8bfb_4a18889db59c.slice/crio-7294d0ac669980970ad07c875a2c2893449e45107ac95521b7afe7a2b64d5dcf WatchSource:0}: Error finding container 7294d0ac669980970ad07c875a2c2893449e45107ac95521b7afe7a2b64d5dcf: Status 404 returned error can't find the container with id 7294d0ac669980970ad07c875a2c2893449e45107ac95521b7afe7a2b64d5dcf
Sep 09 03:17:55.792502 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:55.792476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-52xfk"]
Sep 09 03:17:56.619268 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:56.619221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-52xfk" event={"ID":"41de35a1-e1a6-41b7-8bfb-4a18889db59c","Type":"ContainerStarted","Data":"7294d0ac669980970ad07c875a2c2893449e45107ac95521b7afe7a2b64d5dcf"}
Sep 09 03:17:56.620824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:56.620795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fc79f467b-mq5wm" event={"ID":"17db3411-56eb-4092-bd50-3ce1544e58e4","Type":"ContainerStarted","Data":"b7266ed271196bac2d4679a6d7dd7508e99a43bd86ff72bc6b5c5498be6dce88"}
Sep 09 03:17:56.620949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:56.620838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fc79f467b-mq5wm" event={"ID":"17db3411-56eb-4092-bd50-3ce1544e58e4","Type":"ContainerStarted","Data":"f738880f735b8c2dfccaed951cb715790dfef2c17578470f8cd2dc9cdfe4e823"}
Sep 09 03:17:56.642948 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:56.642909    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6fc79f467b-mq5wm" podStartSLOduration=33.642891856 podStartE2EDuration="33.642891856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:56.641560847 +0000 UTC m=+287.341324806" watchObservedRunningTime="2026-09-09 03:17:56.642891856 +0000 UTC m=+287.342655823"
Sep 09 03:17:57.459311 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.459282    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:57.461818 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.461798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:57.625587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.625525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-52xfk" event={"ID":"41de35a1-e1a6-41b7-8bfb-4a18889db59c","Type":"ContainerStarted","Data":"579180c1c098d8970286b42bb20fc8bebbf3cb54fff173cf6d5fc02f581616b4"}
Sep 09 03:17:57.625587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.625574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-52xfk" event={"ID":"41de35a1-e1a6-41b7-8bfb-4a18889db59c","Type":"ContainerStarted","Data":"e81be67b6fec671e96e2fdd26121e706298170cdd51ada1d4b51a7bb1a703634"}
Sep 09 03:17:57.626030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.625795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:57.626880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.626864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6fc79f467b-mq5wm"
Sep 09 03:17:57.643373 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:57.643333    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-52xfk" podStartSLOduration=251.193368946 podStartE2EDuration="4m12.643321915s" totalImagesPullingTime="1.449952969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:17:57.642917555 +0000 UTC m=+288.342681519" watchObservedRunningTime="2026-09-09 03:17:57.643321915 +0000 UTC m=+288.343085877"
Sep 09 03:17:58.628284 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:17:58.628254    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-52xfk"
Sep 09 03:18:01.984467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:01.984438    2580 patch_prober.go:28] interesting pod/image-registry-55f5846475-gxmg5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 09 03:18:01.984838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:01.984486    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" podUID="0fd1dd10-737a-4a07-9087-50bc04d158d5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:18:05.784097 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:18:05.784054    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fb6s8" podUID="5adb8729-7a58-4cba-9f38-ea9ce848593e"
Sep 09 03:18:08.632523 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:08.632492    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-52xfk"
Sep 09 03:18:09.725727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:09.725704    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 03:18:11.982429 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:11.982404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:18:14.731813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.731783    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-724z9"]
Sep 09 03:18:14.737477 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.737457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.740394 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.740374    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 03:18:14.740495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.740380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 03:18:14.740495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.740430    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z8kcn\""
Sep 09 03:18:14.740495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.740445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 03:18:14.741270 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.741252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 03:18:14.741270 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.741267    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 03:18:14.743467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.743448    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 03:18:14.848280 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-root\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-textfile\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-wtmp\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-metrics-client-ca\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlzqt\" (UniqueName: \"kubernetes.io/projected/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-kube-api-access-zlzqt\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-tls\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.848555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.848531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-sys\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949818 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-metrics-client-ca\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlzqt\" (UniqueName: \"kubernetes.io/projected/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-kube-api-access-zlzqt\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-tls\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-sys\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-root\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-textfile\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.949982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.949970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-wtmp\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-sys\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-root\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-wtmp\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950466 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-textfile\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950466 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-metrics-client-ca\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.950547 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.950526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.952423 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.952402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.952514 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.952439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-node-exporter-tls\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:14.957744 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:14.957723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlzqt\" (UniqueName: \"kubernetes.io/projected/0dacdf99-bfd3-46cd-883a-0b2ee45499c8-kube-api-access-zlzqt\") pod \"node-exporter-724z9\" (UID: \"0dacdf99-bfd3-46cd-883a-0b2ee45499c8\") " pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:15.046918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:15.046851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-724z9"
Sep 09 03:18:15.058954 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:18:15.058933    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dacdf99_bfd3_46cd_883a_0b2ee45499c8.slice/crio-885ec570b7d849f6a32629adcbfa786b9619618a987111c9455998cf47ec0c40 WatchSource:0}: Error finding container 885ec570b7d849f6a32629adcbfa786b9619618a987111c9455998cf47ec0c40: Status 404 returned error can't find the container with id 885ec570b7d849f6a32629adcbfa786b9619618a987111c9455998cf47ec0c40
Sep 09 03:18:15.060972 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:15.060949    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:18:15.685126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:15.685095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-724z9" event={"ID":"0dacdf99-bfd3-46cd-883a-0b2ee45499c8","Type":"ContainerStarted","Data":"885ec570b7d849f6a32629adcbfa786b9619618a987111c9455998cf47ec0c40"}
Sep 09 03:18:16.694430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:16.694359    2580 generic.go:356] "Generic (PLEG): container finished" podID="0dacdf99-bfd3-46cd-883a-0b2ee45499c8" containerID="8605a37d2e8c1efc137eda7af4c8c50bfce7a0eed69a6008ccf16560117ffe1e" exitCode=0
Sep 09 03:18:16.694768 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:16.694448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-724z9" event={"ID":"0dacdf99-bfd3-46cd-883a-0b2ee45499c8","Type":"ContainerDied","Data":"8605a37d2e8c1efc137eda7af4c8c50bfce7a0eed69a6008ccf16560117ffe1e"}
Sep 09 03:18:16.997541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:16.997505    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" podUID="0fd1dd10-737a-4a07-9087-50bc04d158d5" containerName="registry" containerID="cri-o://aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5" gracePeriod=30
Sep 09 03:18:17.214331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.214303    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:18:17.368402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368376    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl79z\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-kube-api-access-sl79z\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368415    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-installation-pull-secrets\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368441    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0fd1dd10-737a-4a07-9087-50bc04d158d5-ca-trust-extracted\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368506    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-certificates\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368551    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-trusted-ca\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-bound-sa-token\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.368727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-image-registry-private-configuration\") pod \"0fd1dd10-737a-4a07-9087-50bc04d158d5\" (UID: \"0fd1dd10-737a-4a07-9087-50bc04d158d5\") "
Sep 09 03:18:17.369124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.368978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:18:17.369124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.369089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:18:17.371055 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.371028    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-kube-api-access-sl79z" (OuterVolumeSpecName: "kube-api-access-sl79z") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "kube-api-access-sl79z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:18:17.371055 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.371045    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:18:17.371238 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.371216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:18:17.371451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.371411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:18:17.371451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.371429    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:18:17.376851 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.376830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fd1dd10-737a-4a07-9087-50bc04d158d5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0fd1dd10-737a-4a07-9087-50bc04d158d5" (UID: "0fd1dd10-737a-4a07-9087-50bc04d158d5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:18:17.470033 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470004    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-trusted-ca\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470033 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470029    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-bound-sa-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470033 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470039    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-image-registry-private-configuration\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470056    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl79z\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-kube-api-access-sl79z\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470066    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0fd1dd10-737a-4a07-9087-50bc04d158d5-installation-pull-secrets\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470074    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-tls\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470082    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0fd1dd10-737a-4a07-9087-50bc04d158d5-ca-trust-extracted\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.470239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.470089    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0fd1dd10-737a-4a07-9087-50bc04d158d5-registry-certificates\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:18:17.699351 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.699277    2580 generic.go:356] "Generic (PLEG): container finished" podID="0fd1dd10-737a-4a07-9087-50bc04d158d5" containerID="aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5" exitCode=0
Sep 09 03:18:17.699351 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.699322    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55f5846475-gxmg5"
Sep 09 03:18:17.699872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.699360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" event={"ID":"0fd1dd10-737a-4a07-9087-50bc04d158d5","Type":"ContainerDied","Data":"aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5"}
Sep 09 03:18:17.699872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.699407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55f5846475-gxmg5" event={"ID":"0fd1dd10-737a-4a07-9087-50bc04d158d5","Type":"ContainerDied","Data":"bd870901f131491b0566bdd5ebe625821381e9527ee0a151e1fd0f8933eb4243"}
Sep 09 03:18:17.699872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.699429    2580 scope.go:117] "RemoveContainer" containerID="aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5"
Sep 09 03:18:17.701746 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.701661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-724z9" event={"ID":"0dacdf99-bfd3-46cd-883a-0b2ee45499c8","Type":"ContainerStarted","Data":"ead2e1fd0973e69a1dd3f5deee8ff22e813c2a3f9fa2defdc56e24a6d8ff6ada"}
Sep 09 03:18:17.701876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.701768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-724z9" event={"ID":"0dacdf99-bfd3-46cd-883a-0b2ee45499c8","Type":"ContainerStarted","Data":"936588ee8f2cf8b961b41c83a5fa4218ffc85acdfa9fe5f7587e9046c3af89e9"}
Sep 09 03:18:17.708031 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.708013    2580 scope.go:117] "RemoveContainer" containerID="aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5"
Sep 09 03:18:17.708282 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:18:17.708263    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5\": container with ID starting with aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5 not found: ID does not exist" containerID="aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5"
Sep 09 03:18:17.708325 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.708291    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5"} err="failed to get container status \"aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5\": rpc error: code = NotFound desc = could not find container \"aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5\": container with ID starting with aacef6f0690633e182fdc1a7e4fbd8b8138ad256bcd357127f4c574d6a5c96d5 not found: ID does not exist"
Sep 09 03:18:17.725231 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.725196    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-724z9" podStartSLOduration=2.418062867 podStartE2EDuration="3.725184165s" totalImagesPullingTime="1.240002458s" totalInitContainerRuntime="67.11884ms" isStatefulPod=true podCreationTimestamp="2026-09-09 03:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:17.724062294 +0000 UTC m=+308.423826257" watchObservedRunningTime="2026-09-09 03:18:17.725184165 +0000 UTC m=+308.424948127"
Sep 09 03:18:17.740483 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.740459    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55f5846475-gxmg5"]
Sep 09 03:18:17.744447 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.744428    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55f5846475-gxmg5"]
Sep 09 03:18:17.786916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:17.786891    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fd1dd10-737a-4a07-9087-50bc04d158d5" path="/var/lib/kubelet/pods/0fd1dd10-737a-4a07-9087-50bc04d158d5/volumes"
Sep 09 03:18:18.824556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.824528    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-859fbb8db7-njr4l"]
Sep 09 03:18:18.824924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.824762    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fd1dd10-737a-4a07-9087-50bc04d158d5" containerName="registry"
Sep 09 03:18:18.824924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.824773    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fd1dd10-737a-4a07-9087-50bc04d158d5" containerName="registry"
Sep 09 03:18:18.824924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.824830    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fd1dd10-737a-4a07-9087-50bc04d158d5" containerName="registry"
Sep 09 03:18:18.829716 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.829698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.832194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.832173    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 09 03:18:18.832328 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.832197    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 09 03:18:18.832328 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.832217    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hf4p9\""
Sep 09 03:18:18.832419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.832323    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-br5r0lk3dnhru\""
Sep 09 03:18:18.833174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.833158    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 09 03:18:18.833223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.833190    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 09 03:18:18.833335 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.833320    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 09 03:18:18.841056 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.841036    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-859fbb8db7-njr4l"]
Sep 09 03:18:18.981395 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/059dee7b-d3d5-4309-afc3-678f05768f20-metrics-client-ca\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-tls\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-grpc-tls\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:18.981748 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:18.981650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6764t\" (UniqueName: \"kubernetes.io/projected/059dee7b-d3d5-4309-afc3-678f05768f20-kube-api-access-6764t\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.026335 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.026305    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d774ddff9-6qgs6"]
Sep 09 03:18:19.029294 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.029281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.031769 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.031748    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eud1j0tv2e2pc\""
Sep 09 03:18:19.032052 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.032036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wcrmf\""
Sep 09 03:18:19.032166 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.032149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 03:18:19.032246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.032160    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 03:18:19.032647 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.032634    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 03:18:19.032898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.032884    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 03:18:19.044043 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.044026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d774ddff9-6qgs6"]
Sep 09 03:18:19.082692 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082692 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082692 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-grpc-tls\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6764t\" (UniqueName: \"kubernetes.io/projected/059dee7b-d3d5-4309-afc3-678f05768f20-kube-api-access-6764t\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/059dee7b-d3d5-4309-afc3-678f05768f20-metrics-client-ca\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.082947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.082937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-tls\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.083873 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.083838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/059dee7b-d3d5-4309-afc3-678f05768f20-metrics-client-ca\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.085402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.085378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.085516 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.085386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.085626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.085574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.085626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.085599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.086174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.086156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-grpc-tls\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.086278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.086218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/059dee7b-d3d5-4309-afc3-678f05768f20-secret-thanos-querier-tls\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.101439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.101416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6764t\" (UniqueName: \"kubernetes.io/projected/059dee7b-d3d5-4309-afc3-678f05768f20-kube-api-access-6764t\") pod \"thanos-querier-859fbb8db7-njr4l\" (UID: \"059dee7b-d3d5-4309-afc3-678f05768f20\") " pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.139240 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.139223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:19.183499 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f6d3b4d-176d-4362-8b19-72568f95f938-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.183614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd624\" (UniqueName: \"kubernetes.io/projected/0f6d3b4d-176d-4362-8b19-72568f95f938-kube-api-access-pd624\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.183614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-secret-metrics-server-tls\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.183824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-secret-metrics-server-client-certs\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.183824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f6d3b4d-176d-4362-8b19-72568f95f938-audit-log\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.183824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f6d3b4d-176d-4362-8b19-72568f95f938-metrics-server-audit-profiles\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.183824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.183762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-client-ca-bundle\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284603 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f6d3b4d-176d-4362-8b19-72568f95f938-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284771 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd624\" (UniqueName: \"kubernetes.io/projected/0f6d3b4d-176d-4362-8b19-72568f95f938-kube-api-access-pd624\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284771 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-secret-metrics-server-tls\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284771 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-secret-metrics-server-client-certs\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284771 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f6d3b4d-176d-4362-8b19-72568f95f938-audit-log\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284979 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f6d3b4d-176d-4362-8b19-72568f95f938-metrics-server-audit-profiles\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.284979 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.284841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-client-ca-bundle\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.285228 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.285200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f6d3b4d-176d-4362-8b19-72568f95f938-audit-log\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.285355 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.285333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f6d3b4d-176d-4362-8b19-72568f95f938-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.285981 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.285953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f6d3b4d-176d-4362-8b19-72568f95f938-metrics-server-audit-profiles\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.287106 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.287080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-secret-metrics-server-tls\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.287294 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.287277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-secret-metrics-server-client-certs\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.287386 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.287369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f6d3b4d-176d-4362-8b19-72568f95f938-client-ca-bundle\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.293036 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.293015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd624\" (UniqueName: \"kubernetes.io/projected/0f6d3b4d-176d-4362-8b19-72568f95f938-kube-api-access-pd624\") pod \"metrics-server-5d774ddff9-6qgs6\" (UID: \"0f6d3b4d-176d-4362-8b19-72568f95f938\") " pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.338020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.337968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:19.421093 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.421009    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-859fbb8db7-njr4l"]
Sep 09 03:18:19.424784 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:18:19.424753    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059dee7b_d3d5_4309_afc3_678f05768f20.slice/crio-c4d9d9bef00ed3c4c061e526268ed9d4c3e52593cf58874c173b5cdfc833260c WatchSource:0}: Error finding container c4d9d9bef00ed3c4c061e526268ed9d4c3e52593cf58874c173b5cdfc833260c: Status 404 returned error can't find the container with id c4d9d9bef00ed3c4c061e526268ed9d4c3e52593cf58874c173b5cdfc833260c
Sep 09 03:18:19.487496 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.487469    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"]
Sep 09 03:18:19.491630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.491613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.493873 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.493834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 03:18:19.493986 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.493936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2c9t8\""
Sep 09 03:18:19.504700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.504664    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"]
Sep 09 03:18:19.588742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.588665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kjfw\" (UniqueName: \"kubernetes.io/projected/8dee1962-43ed-44c1-ac6c-7e5156757517-kube-api-access-2kjfw\") pod \"monitoring-plugin-85bbbbbf4-7kkqm\" (UID: \"8dee1962-43ed-44c1-ac6c-7e5156757517\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.588835 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.588740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8dee1962-43ed-44c1-ac6c-7e5156757517-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-7kkqm\" (UID: \"8dee1962-43ed-44c1-ac6c-7e5156757517\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.605945 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.605915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d774ddff9-6qgs6"]
Sep 09 03:18:19.608981 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:18:19.608957    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6d3b4d_176d_4362_8b19_72568f95f938.slice/crio-71cb72acc103f213c95e340fa74a5ab5db6f9c73f8f326609bcfc916063ded08 WatchSource:0}: Error finding container 71cb72acc103f213c95e340fa74a5ab5db6f9c73f8f326609bcfc916063ded08: Status 404 returned error can't find the container with id 71cb72acc103f213c95e340fa74a5ab5db6f9c73f8f326609bcfc916063ded08
Sep 09 03:18:19.689846 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.689820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8dee1962-43ed-44c1-ac6c-7e5156757517-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-7kkqm\" (UID: \"8dee1962-43ed-44c1-ac6c-7e5156757517\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.689938 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.689889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kjfw\" (UniqueName: \"kubernetes.io/projected/8dee1962-43ed-44c1-ac6c-7e5156757517-kube-api-access-2kjfw\") pod \"monitoring-plugin-85bbbbbf4-7kkqm\" (UID: \"8dee1962-43ed-44c1-ac6c-7e5156757517\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.692196 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.692179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8dee1962-43ed-44c1-ac6c-7e5156757517-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-7kkqm\" (UID: \"8dee1962-43ed-44c1-ac6c-7e5156757517\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.698210 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.698192    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kjfw\" (UniqueName: \"kubernetes.io/projected/8dee1962-43ed-44c1-ac6c-7e5156757517-kube-api-access-2kjfw\") pod \"monitoring-plugin-85bbbbbf4-7kkqm\" (UID: \"8dee1962-43ed-44c1-ac6c-7e5156757517\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.710827 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.710807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"c4d9d9bef00ed3c4c061e526268ed9d4c3e52593cf58874c173b5cdfc833260c"}
Sep 09 03:18:19.712047 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.712028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6" event={"ID":"0f6d3b4d-176d-4362-8b19-72568f95f938","Type":"ContainerStarted","Data":"71cb72acc103f213c95e340fa74a5ab5db6f9c73f8f326609bcfc916063ded08"}
Sep 09 03:18:19.784627 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.784604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:18:19.800999 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.800979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:19.964616 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.964587    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"]
Sep 09 03:18:19.967909 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.967889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:19.970587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.970528    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 03:18:19.970587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.970548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 03:18:19.970587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.970583    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 03:18:19.970885 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.970864    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 03:18:19.971061 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.971045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 03:18:19.971183 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.971166    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-96r98\""
Sep 09 03:18:19.976428 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.976120    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 03:18:19.986998 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:19.986979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"]
Sep 09 03:18:20.075242 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.075216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"]
Sep 09 03:18:20.078317 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:18:20.078290    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dee1962_43ed_44c1_ac6c_7e5156757517.slice/crio-c3390098d57b42e30ab36c9712f0256f850b3cc5d56008a5adcbc1abdd0fc673 WatchSource:0}: Error finding container c3390098d57b42e30ab36c9712f0256f850b3cc5d56008a5adcbc1abdd0fc673: Status 404 returned error can't find the container with id c3390098d57b42e30ab36c9712f0256f850b3cc5d56008a5adcbc1abdd0fc673
Sep 09 03:18:20.093643 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.093748 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.093748 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-secret-telemeter-client\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.093872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx5fh\" (UniqueName: \"kubernetes.io/projected/b2452083-20e5-4038-9739-b3c82ea97722-kube-api-access-sx5fh\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.093872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-telemeter-client-tls\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.093975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-serving-certs-ca-bundle\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.093975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-federate-client-tls\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.094085 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.093988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-metrics-client-ca\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-federate-client-tls\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-metrics-client-ca\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-secret-telemeter-client\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx5fh\" (UniqueName: \"kubernetes.io/projected/b2452083-20e5-4038-9739-b3c82ea97722-kube-api-access-sx5fh\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-telemeter-client-tls\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.194577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.194523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-serving-certs-ca-bundle\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.195527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.195142    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-metrics-client-ca\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.195527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.195179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-serving-certs-ca-bundle\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.195729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.195638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2452083-20e5-4038-9739-b3c82ea97722-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.197482 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.197453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-federate-client-tls\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.198041 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.197997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-telemeter-client-tls\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.198159 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.198079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-secret-telemeter-client\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.198306 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.198286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2452083-20e5-4038-9739-b3c82ea97722-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.204148 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.204119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx5fh\" (UniqueName: \"kubernetes.io/projected/b2452083-20e5-4038-9739-b3c82ea97722-kube-api-access-sx5fh\") pod \"telemeter-client-6d4759d4b5-s9wkw\" (UID: \"b2452083-20e5-4038-9739-b3c82ea97722\") " pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.278980 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.278955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"
Sep 09 03:18:20.564502 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.564461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw"]
Sep 09 03:18:20.717511 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:20.717444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm" event={"ID":"8dee1962-43ed-44c1-ac6c-7e5156757517","Type":"ContainerStarted","Data":"c3390098d57b42e30ab36c9712f0256f850b3cc5d56008a5adcbc1abdd0fc673"}
Sep 09 03:18:21.056585 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:18:21.056554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2452083_20e5_4038_9739_b3c82ea97722.slice/crio-840df1689110e4f67fd13bd3733520c42e1496b81a296b01786dac863d590d1c WatchSource:0}: Error finding container 840df1689110e4f67fd13bd3733520c42e1496b81a296b01786dac863d590d1c: Status 404 returned error can't find the container with id 840df1689110e4f67fd13bd3733520c42e1496b81a296b01786dac863d590d1c
Sep 09 03:18:21.724369 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:21.724278    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw" event={"ID":"b2452083-20e5-4038-9739-b3c82ea97722","Type":"ContainerStarted","Data":"840df1689110e4f67fd13bd3733520c42e1496b81a296b01786dac863d590d1c"}
Sep 09 03:18:22.730465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.730431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6" event={"ID":"0f6d3b4d-176d-4362-8b19-72568f95f938","Type":"ContainerStarted","Data":"e768f903d9fb2fdadaa1ab1bfc95980db330239fe76a522efa860b8e58f030d2"}
Sep 09 03:18:22.733178 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.733145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"aeabdd177f4eeaf51b5b43ad0b559f4138facd92f4359692fe9e16da0a0f2927"}
Sep 09 03:18:22.733315 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.733225    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"4d860c303304b9748d671044a61bfac5763c425e9561f2c3b67fd70bdcbd9272"}
Sep 09 03:18:22.733315 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.733268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"8b0123bb60440684719b622c301bab9d3253c5bd859534824f8c366750c37713"}
Sep 09 03:18:22.734666 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.734638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm" event={"ID":"8dee1962-43ed-44c1-ac6c-7e5156757517","Type":"ContainerStarted","Data":"3dbc12b271f1a8baecbefcac6c86211587b3855fc1594e0a45516e7941cd727c"}
Sep 09 03:18:22.734881 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.734855    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:22.740260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.740240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm"
Sep 09 03:18:22.750700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.750647    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6" podStartSLOduration=1.52440357 podStartE2EDuration="3.750634315s" totalImagesPullingTime="2.226230745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:22.749481506 +0000 UTC m=+313.449245468" watchObservedRunningTime="2026-09-09 03:18:22.750634315 +0000 UTC m=+313.450398301"
Sep 09 03:18:22.764848 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:22.764813    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-7kkqm" podStartSLOduration=1.954834815 podStartE2EDuration="3.764801904s" totalImagesPullingTime="1.809967089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:22.76392944 +0000 UTC m=+313.463693404" watchObservedRunningTime="2026-09-09 03:18:22.764801904 +0000 UTC m=+313.464565865"
Sep 09 03:18:23.742727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:23.742690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"cebf3926743ec7ef1b5d8b4087c97f44c214a364d8d4992b3b65c2b34d4866df"}
Sep 09 03:18:23.743196 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:23.742777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"59b42c05a5430c8b1776eec03ff90a4cadb3a615963264884045f2f9a0e32cfc"}
Sep 09 03:18:23.743196 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:23.742819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" event={"ID":"059dee7b-d3d5-4309-afc3-678f05768f20","Type":"ContainerStarted","Data":"2a909dbe46a4b80d1cfddd2ea3ff94a3d2356a24de927b65a8c2b24ef273dbac"}
Sep 09 03:18:23.743196 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:23.742890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:23.744437 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:23.744411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw" event={"ID":"b2452083-20e5-4038-9739-b3c82ea97722","Type":"ContainerStarted","Data":"b0fe3c8d70093f2204bb1733198d889ef6ecd668491a8add0d805894fd8eb776"}
Sep 09 03:18:23.767908 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:23.767869    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l" podStartSLOduration=2.206524638 podStartE2EDuration="5.767838417s" totalImagesPullingTime="3.561313779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:18:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:23.766370671 +0000 UTC m=+314.466134634" watchObservedRunningTime="2026-09-09 03:18:23.767838417 +0000 UTC m=+314.467602422"
Sep 09 03:18:24.749664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:24.749627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw" event={"ID":"b2452083-20e5-4038-9739-b3c82ea97722","Type":"ContainerStarted","Data":"e0811e33717cb6520a42595fc4e659f98a11ca7ea5c3f28f64d6efc27648743f"}
Sep 09 03:18:24.750107 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:24.749748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw" event={"ID":"b2452083-20e5-4038-9739-b3c82ea97722","Type":"ContainerStarted","Data":"d8c2299f59c02f12022f69de6820961fc4d50458b13d0a29e829e4f7a7d36984"}
Sep 09 03:18:24.772921 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:24.772876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d4759d4b5-s9wkw" podStartSLOduration=2.799944412 podStartE2EDuration="5.772864213s" totalImagesPullingTime="2.972919801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:18:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:18:24.772741329 +0000 UTC m=+315.472505291" watchObservedRunningTime="2026-09-09 03:18:24.772864213 +0000 UTC m=+315.472628174"
Sep 09 03:18:29.756161 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:29.756133    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-859fbb8db7-njr4l"
Sep 09 03:18:39.339060 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:39.339028    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:39.339444 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:39.339087    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:46.824570 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:46.824545    2580 generic.go:356] "Generic (PLEG): container finished" podID="b4247b76-d174-41a9-a607-ecc4994781a5" containerID="5667e49b28a3fdd970ff2b172c4218907f70d1541a1b55f4746026d62c5830d6" exitCode=0
Sep 09 03:18:46.824914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:46.824620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc" event={"ID":"b4247b76-d174-41a9-a607-ecc4994781a5","Type":"ContainerDied","Data":"5667e49b28a3fdd970ff2b172c4218907f70d1541a1b55f4746026d62c5830d6"}
Sep 09 03:18:46.824977 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:46.824924    2580 scope.go:117] "RemoveContainer" containerID="5667e49b28a3fdd970ff2b172c4218907f70d1541a1b55f4746026d62c5830d6"
Sep 09 03:18:47.829426 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:47.829395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-x9czc" event={"ID":"b4247b76-d174-41a9-a607-ecc4994781a5","Type":"ContainerStarted","Data":"498433f08b001d2d73b92130c0b833a475def4992fe9a813ca5734f846dd53fb"}
Sep 09 03:18:59.343843 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:59.343805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:18:59.347551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:18:59.347525    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d774ddff9-6qgs6"
Sep 09 03:19:23.761361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:23.761333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:19:23.763590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:23.763573    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5adb8729-7a58-4cba-9f38-ea9ce848593e-metrics-certs\") pod \"network-metrics-daemon-fb6s8\" (UID: \"5adb8729-7a58-4cba-9f38-ea9ce848593e\") " pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:19:23.987881 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:23.987850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rkcrq\""
Sep 09 03:19:23.996070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:23.996050    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fb6s8"
Sep 09 03:19:24.267891 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:24.267853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fb6s8"]
Sep 09 03:19:24.270810 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:19:24.270783    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adb8729_7a58_4cba_9f38_ea9ce848593e.slice/crio-0cf2647d0509ce29056a2b976d23a4d55c226d983877da6e6d054f5c3e971399 WatchSource:0}: Error finding container 0cf2647d0509ce29056a2b976d23a4d55c226d983877da6e6d054f5c3e971399: Status 404 returned error can't find the container with id 0cf2647d0509ce29056a2b976d23a4d55c226d983877da6e6d054f5c3e971399
Sep 09 03:19:24.972174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:24.972145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fb6s8" event={"ID":"5adb8729-7a58-4cba-9f38-ea9ce848593e","Type":"ContainerStarted","Data":"0cf2647d0509ce29056a2b976d23a4d55c226d983877da6e6d054f5c3e971399"}
Sep 09 03:19:25.977703 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:25.977659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fb6s8" event={"ID":"5adb8729-7a58-4cba-9f38-ea9ce848593e","Type":"ContainerStarted","Data":"e741812a40b84a258b59d230d6961f8c408f4e38be014416888b364bba100c13"}
Sep 09 03:19:25.978070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:25.977722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fb6s8" event={"ID":"5adb8729-7a58-4cba-9f38-ea9ce848593e","Type":"ContainerStarted","Data":"fbd4777604c7e96b43b5730eba5fda87e85c473793c1a493294e12449ed64adf"}
Sep 09 03:19:25.994186 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:19:25.994143    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fb6s8" podStartSLOduration=375.101140788 podStartE2EDuration="6m15.994129169s" totalImagesPullingTime="892.988381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:19:25.993406054 +0000 UTC m=+376.693170016" watchObservedRunningTime="2026-09-09 03:19:25.994129169 +0000 UTC m=+376.693893125"
Sep 09 03:20:32.469614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.469535    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"]
Sep 09 03:20:32.472813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.472796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.475327 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.475303    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 03:20:32.475437 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.475307    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 03:20:32.476400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.476382    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qsrqj\""
Sep 09 03:20:32.484044 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.483917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"]
Sep 09 03:20:32.536239 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.536203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.536368 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.536279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twphw\" (UniqueName: \"kubernetes.io/projected/59f4bed0-7170-40fb-af6f-37652c9a21bd-kube-api-access-twphw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.536368 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.536323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.636823 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.636791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.636967 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.636853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.636967 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.636934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twphw\" (UniqueName: \"kubernetes.io/projected/59f4bed0-7170-40fb-af6f-37652c9a21bd-kube-api-access-twphw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.637156 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.637137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.637214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.637200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.647655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.647631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twphw\" (UniqueName: \"kubernetes.io/projected/59f4bed0-7170-40fb-af6f-37652c9a21bd-kube-api-access-twphw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:32.781690 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:32.781620    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:33.050776 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:33.050751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"]
Sep 09 03:20:33.053323 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:20:33.053294    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f4bed0_7170_40fb_af6f_37652c9a21bd.slice/crio-d270436576e7f0966962ae556b5ad55502e12f73108afe14ab7e67000a03936e WatchSource:0}: Error finding container d270436576e7f0966962ae556b5ad55502e12f73108afe14ab7e67000a03936e: Status 404 returned error can't find the container with id d270436576e7f0966962ae556b5ad55502e12f73108afe14ab7e67000a03936e
Sep 09 03:20:33.219555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:33.219529    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c" event={"ID":"59f4bed0-7170-40fb-af6f-37652c9a21bd","Type":"ContainerStarted","Data":"d270436576e7f0966962ae556b5ad55502e12f73108afe14ab7e67000a03936e"}
Sep 09 03:20:39.246176 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:39.246148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c" event={"ID":"59f4bed0-7170-40fb-af6f-37652c9a21bd","Type":"ContainerStarted","Data":"cf5c035fad9d8158a1b4db9111847fc3ebfebaab590b243a9675a1b152e57839"}
Sep 09 03:20:40.251348 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:40.251312    2580 generic.go:356] "Generic (PLEG): container finished" podID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerID="cf5c035fad9d8158a1b4db9111847fc3ebfebaab590b243a9675a1b152e57839" exitCode=0
Sep 09 03:20:40.251823 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:40.251400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c" event={"ID":"59f4bed0-7170-40fb-af6f-37652c9a21bd","Type":"ContainerDied","Data":"cf5c035fad9d8158a1b4db9111847fc3ebfebaab590b243a9675a1b152e57839"}
Sep 09 03:20:43.264344 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:43.264314    2580 generic.go:356] "Generic (PLEG): container finished" podID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerID="510438468a97357e987b15b3a194a9a65c67c5168e8a600e2d9290eb6e3d5194" exitCode=0
Sep 09 03:20:43.264741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:43.264373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c" event={"ID":"59f4bed0-7170-40fb-af6f-37652c9a21bd","Type":"ContainerDied","Data":"510438468a97357e987b15b3a194a9a65c67c5168e8a600e2d9290eb6e3d5194"}
Sep 09 03:20:50.295005 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:50.294963    2580 generic.go:356] "Generic (PLEG): container finished" podID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerID="4fe7057d553b3c494f423331ec78580665d3c726d8d94c4aa4ae051083a19c3c" exitCode=0
Sep 09 03:20:50.295337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:50.295015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c" event={"ID":"59f4bed0-7170-40fb-af6f-37652c9a21bd","Type":"ContainerDied","Data":"4fe7057d553b3c494f423331ec78580665d3c726d8d94c4aa4ae051083a19c3c"}
Sep 09 03:20:51.426138 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.426110    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:51.489704 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.489662    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-bundle\") pod \"59f4bed0-7170-40fb-af6f-37652c9a21bd\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") "
Sep 09 03:20:51.489803 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.489743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twphw\" (UniqueName: \"kubernetes.io/projected/59f4bed0-7170-40fb-af6f-37652c9a21bd-kube-api-access-twphw\") pod \"59f4bed0-7170-40fb-af6f-37652c9a21bd\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") "
Sep 09 03:20:51.489803 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.489766    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-util\") pod \"59f4bed0-7170-40fb-af6f-37652c9a21bd\" (UID: \"59f4bed0-7170-40fb-af6f-37652c9a21bd\") "
Sep 09 03:20:51.490406 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.490344    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-bundle" (OuterVolumeSpecName: "bundle") pod "59f4bed0-7170-40fb-af6f-37652c9a21bd" (UID: "59f4bed0-7170-40fb-af6f-37652c9a21bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:20:51.491938 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.491919    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f4bed0-7170-40fb-af6f-37652c9a21bd-kube-api-access-twphw" (OuterVolumeSpecName: "kube-api-access-twphw") pod "59f4bed0-7170-40fb-af6f-37652c9a21bd" (UID: "59f4bed0-7170-40fb-af6f-37652c9a21bd"). InnerVolumeSpecName "kube-api-access-twphw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:20:51.493729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.493709    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-util" (OuterVolumeSpecName: "util") pod "59f4bed0-7170-40fb-af6f-37652c9a21bd" (UID: "59f4bed0-7170-40fb-af6f-37652c9a21bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:20:51.590199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.590156    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:20:51.590199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.590175    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twphw\" (UniqueName: \"kubernetes.io/projected/59f4bed0-7170-40fb-af6f-37652c9a21bd-kube-api-access-twphw\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:20:51.590199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:51.590184    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/59f4bed0-7170-40fb-af6f-37652c9a21bd-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:20:52.304783 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:52.304760    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c"
Sep 09 03:20:52.304942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:52.304759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754095v7c" event={"ID":"59f4bed0-7170-40fb-af6f-37652c9a21bd","Type":"ContainerDied","Data":"d270436576e7f0966962ae556b5ad55502e12f73108afe14ab7e67000a03936e"}
Sep 09 03:20:52.304942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:52.304883    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d270436576e7f0966962ae556b5ad55502e12f73108afe14ab7e67000a03936e"
Sep 09 03:20:59.881463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881430    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"]
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881752    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="extract"
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881765    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="extract"
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881776    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="util"
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881781    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="util"
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881789    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="pull"
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881794    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="pull"
Sep 09 03:20:59.881914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.881845    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f4bed0-7170-40fb-af6f-37652c9a21bd" containerName="extract"
Sep 09 03:20:59.885363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.885348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:20:59.887975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.887950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 03:20:59.888161 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.888149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qsrqj\""
Sep 09 03:20:59.888903 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.888884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 03:20:59.896850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.896829    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"]
Sep 09 03:20:59.948282 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.948257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npj9n\" (UniqueName: \"kubernetes.io/projected/7a90c07f-c513-4881-8a59-e98f2fd9d687-kube-api-access-npj9n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:20:59.948411 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.948297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:20:59.948411 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:20:59.948339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.048807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.048779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npj9n\" (UniqueName: \"kubernetes.io/projected/7a90c07f-c513-4881-8a59-e98f2fd9d687-kube-api-access-npj9n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.048936 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.048821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.048936 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.048858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.049227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.049209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.049298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.049227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.058787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.058765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npj9n\" (UniqueName: \"kubernetes.io/projected/7a90c07f-c513-4881-8a59-e98f2fd9d687-kube-api-access-npj9n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.194586 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.194524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:00.297794 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.297764    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s8dmr"]
Sep 09 03:21:00.299763 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.299744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.302873 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.302851    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 03:21:00.303024 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.302998    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 03:21:00.303222 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.303207    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bz4bs\""
Sep 09 03:21:00.312031 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.312010    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s8dmr"]
Sep 09 03:21:00.453931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.453866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n26zg\" (UniqueName: \"kubernetes.io/projected/fca01fde-f6e1-4195-b20f-bdc35fc1ac07-kube-api-access-n26zg\") pod \"cert-manager-webhook-6486ffd997-s8dmr\" (UID: \"fca01fde-f6e1-4195-b20f-bdc35fc1ac07\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.453931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.453898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fca01fde-f6e1-4195-b20f-bdc35fc1ac07-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s8dmr\" (UID: \"fca01fde-f6e1-4195-b20f-bdc35fc1ac07\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.464591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.464558    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"]
Sep 09 03:21:00.466718 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:00.466659    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a90c07f_c513_4881_8a59_e98f2fd9d687.slice/crio-6d1bd4748f35adc98495542aae3e22e67aa2833e9971c71b98b468786559a774 WatchSource:0}: Error finding container 6d1bd4748f35adc98495542aae3e22e67aa2833e9971c71b98b468786559a774: Status 404 returned error can't find the container with id 6d1bd4748f35adc98495542aae3e22e67aa2833e9971c71b98b468786559a774
Sep 09 03:21:00.554492 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.554466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n26zg\" (UniqueName: \"kubernetes.io/projected/fca01fde-f6e1-4195-b20f-bdc35fc1ac07-kube-api-access-n26zg\") pod \"cert-manager-webhook-6486ffd997-s8dmr\" (UID: \"fca01fde-f6e1-4195-b20f-bdc35fc1ac07\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.554603 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.554503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fca01fde-f6e1-4195-b20f-bdc35fc1ac07-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s8dmr\" (UID: \"fca01fde-f6e1-4195-b20f-bdc35fc1ac07\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.569876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.569855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fca01fde-f6e1-4195-b20f-bdc35fc1ac07-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s8dmr\" (UID: \"fca01fde-f6e1-4195-b20f-bdc35fc1ac07\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.569963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.569906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n26zg\" (UniqueName: \"kubernetes.io/projected/fca01fde-f6e1-4195-b20f-bdc35fc1ac07-kube-api-access-n26zg\") pod \"cert-manager-webhook-6486ffd997-s8dmr\" (UID: \"fca01fde-f6e1-4195-b20f-bdc35fc1ac07\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.609921 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.609899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:00.887365 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:00.887231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s8dmr"]
Sep 09 03:21:00.889713 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:00.889685    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfca01fde_f6e1_4195_b20f_bdc35fc1ac07.slice/crio-8b8ce9e4a24dd4cc5cd3bce9a2399f95fcaa2f78f9eee3430794bc851d9004a2 WatchSource:0}: Error finding container 8b8ce9e4a24dd4cc5cd3bce9a2399f95fcaa2f78f9eee3430794bc851d9004a2: Status 404 returned error can't find the container with id 8b8ce9e4a24dd4cc5cd3bce9a2399f95fcaa2f78f9eee3430794bc851d9004a2
Sep 09 03:21:01.344689 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.344643    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerID="e6d704c45167f2bc7b80a6e69deffab21db4c936541af421e95c7ff1982fed55" exitCode=0
Sep 09 03:21:01.344916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.344715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4" event={"ID":"7a90c07f-c513-4881-8a59-e98f2fd9d687","Type":"ContainerDied","Data":"e6d704c45167f2bc7b80a6e69deffab21db4c936541af421e95c7ff1982fed55"}
Sep 09 03:21:01.344916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.344803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4" event={"ID":"7a90c07f-c513-4881-8a59-e98f2fd9d687","Type":"ContainerStarted","Data":"6d1bd4748f35adc98495542aae3e22e67aa2833e9971c71b98b468786559a774"}
Sep 09 03:21:01.346121 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.346101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr" event={"ID":"fca01fde-f6e1-4195-b20f-bdc35fc1ac07","Type":"ContainerStarted","Data":"8b8ce9e4a24dd4cc5cd3bce9a2399f95fcaa2f78f9eee3430794bc851d9004a2"}
Sep 09 03:21:01.693857 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.693786    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"]
Sep 09 03:21:01.696067 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.696043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:01.698880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.698859    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-25fx9\""
Sep 09 03:21:01.714346 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.714322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"]
Sep 09 03:21:01.763418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.763387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdg49\" (UniqueName: \"kubernetes.io/projected/175f4e53-499f-4d6a-9018-3f1cfdfd7c13-kube-api-access-mdg49\") pod \"cert-manager-cainjector-559cdd8c87-zlfq2\" (UID: \"175f4e53-499f-4d6a-9018-3f1cfdfd7c13\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:01.763535 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.763483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/175f4e53-499f-4d6a-9018-3f1cfdfd7c13-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zlfq2\" (UID: \"175f4e53-499f-4d6a-9018-3f1cfdfd7c13\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:01.864211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.864180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdg49\" (UniqueName: \"kubernetes.io/projected/175f4e53-499f-4d6a-9018-3f1cfdfd7c13-kube-api-access-mdg49\") pod \"cert-manager-cainjector-559cdd8c87-zlfq2\" (UID: \"175f4e53-499f-4d6a-9018-3f1cfdfd7c13\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:01.864372 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.864279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/175f4e53-499f-4d6a-9018-3f1cfdfd7c13-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zlfq2\" (UID: \"175f4e53-499f-4d6a-9018-3f1cfdfd7c13\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:01.875313 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.875285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdg49\" (UniqueName: \"kubernetes.io/projected/175f4e53-499f-4d6a-9018-3f1cfdfd7c13-kube-api-access-mdg49\") pod \"cert-manager-cainjector-559cdd8c87-zlfq2\" (UID: \"175f4e53-499f-4d6a-9018-3f1cfdfd7c13\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:01.875417 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:01.875355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/175f4e53-499f-4d6a-9018-3f1cfdfd7c13-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zlfq2\" (UID: \"175f4e53-499f-4d6a-9018-3f1cfdfd7c13\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:02.006065 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:02.006033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"
Sep 09 03:21:02.290942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:02.290852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2"]
Sep 09 03:21:02.294184 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:02.294142    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod175f4e53_499f_4d6a_9018_3f1cfdfd7c13.slice/crio-5ce9bbd5ff43bf4c9df41cdf31d7d3e75e9963f3a570d75defc0eb8ab878b13c WatchSource:0}: Error finding container 5ce9bbd5ff43bf4c9df41cdf31d7d3e75e9963f3a570d75defc0eb8ab878b13c: Status 404 returned error can't find the container with id 5ce9bbd5ff43bf4c9df41cdf31d7d3e75e9963f3a570d75defc0eb8ab878b13c
Sep 09 03:21:02.354051 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:02.354005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2" event={"ID":"175f4e53-499f-4d6a-9018-3f1cfdfd7c13","Type":"ContainerStarted","Data":"5ce9bbd5ff43bf4c9df41cdf31d7d3e75e9963f3a570d75defc0eb8ab878b13c"}
Sep 09 03:21:04.365698 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.365643    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerID="3242204a5375e1f0e56c5f10413cb0d4ec3fd59fb8d1b4bc64d000fb8857e5e4" exitCode=0
Sep 09 03:21:04.366038 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.365699    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4" event={"ID":"7a90c07f-c513-4881-8a59-e98f2fd9d687","Type":"ContainerDied","Data":"3242204a5375e1f0e56c5f10413cb0d4ec3fd59fb8d1b4bc64d000fb8857e5e4"}
Sep 09 03:21:04.367476 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.367452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr" event={"ID":"fca01fde-f6e1-4195-b20f-bdc35fc1ac07","Type":"ContainerStarted","Data":"5255e7c63f0df85c3e0b964b768a63bb734814c41d23c69b63c338147d16652b"}
Sep 09 03:21:04.367588 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.367514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:04.369091 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.369069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2" event={"ID":"175f4e53-499f-4d6a-9018-3f1cfdfd7c13","Type":"ContainerStarted","Data":"446950a430a08692d225f8ae8b9bb136955b8eee8f314e73612d4535dbe6b073"}
Sep 09 03:21:04.436162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.436121    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr" podStartSLOduration=1.202609892 podStartE2EDuration="4.436105106s" totalImagesPullingTime="3.233495214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:21:04.433455853 +0000 UTC m=+475.133219815" watchObservedRunningTime="2026-09-09 03:21:04.436105106 +0000 UTC m=+475.135869069"
Sep 09 03:21:04.464101 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:04.464050    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zlfq2" podStartSLOduration=1.60961289 podStartE2EDuration="3.46403255s" totalImagesPullingTime="1.85441966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:21:04.463294984 +0000 UTC m=+475.163058936" watchObservedRunningTime="2026-09-09 03:21:04.46403255 +0000 UTC m=+475.163796504"
Sep 09 03:21:05.375395 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:05.375360    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerID="a37097dfcfac00d856f9854e0aa17af079a25a283373a616e5d8960fd1e3e32a" exitCode=0
Sep 09 03:21:05.375783 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:05.375449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4" event={"ID":"7a90c07f-c513-4881-8a59-e98f2fd9d687","Type":"ContainerDied","Data":"a37097dfcfac00d856f9854e0aa17af079a25a283373a616e5d8960fd1e3e32a"}
Sep 09 03:21:06.499357 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.499337    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:06.602202 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.602180    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npj9n\" (UniqueName: \"kubernetes.io/projected/7a90c07f-c513-4881-8a59-e98f2fd9d687-kube-api-access-npj9n\") pod \"7a90c07f-c513-4881-8a59-e98f2fd9d687\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") "
Sep 09 03:21:06.602302 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.602211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-util\") pod \"7a90c07f-c513-4881-8a59-e98f2fd9d687\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") "
Sep 09 03:21:06.602302 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.602240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-bundle\") pod \"7a90c07f-c513-4881-8a59-e98f2fd9d687\" (UID: \"7a90c07f-c513-4881-8a59-e98f2fd9d687\") "
Sep 09 03:21:06.602614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.602590    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-bundle" (OuterVolumeSpecName: "bundle") pod "7a90c07f-c513-4881-8a59-e98f2fd9d687" (UID: "7a90c07f-c513-4881-8a59-e98f2fd9d687"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:21:06.604307 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.604279    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a90c07f-c513-4881-8a59-e98f2fd9d687-kube-api-access-npj9n" (OuterVolumeSpecName: "kube-api-access-npj9n") pod "7a90c07f-c513-4881-8a59-e98f2fd9d687" (UID: "7a90c07f-c513-4881-8a59-e98f2fd9d687"). InnerVolumeSpecName "kube-api-access-npj9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:21:06.608296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.608272    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-util" (OuterVolumeSpecName: "util") pod "7a90c07f-c513-4881-8a59-e98f2fd9d687" (UID: "7a90c07f-c513-4881-8a59-e98f2fd9d687"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:21:06.703079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.703019    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:06.703079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.703042    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npj9n\" (UniqueName: \"kubernetes.io/projected/7a90c07f-c513-4881-8a59-e98f2fd9d687-kube-api-access-npj9n\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:06.703079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:06.703052    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a90c07f-c513-4881-8a59-e98f2fd9d687-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:07.386259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:07.386233    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4"
Sep 09 03:21:07.386259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:07.386244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ghc4" event={"ID":"7a90c07f-c513-4881-8a59-e98f2fd9d687","Type":"ContainerDied","Data":"6d1bd4748f35adc98495542aae3e22e67aa2833e9971c71b98b468786559a774"}
Sep 09 03:21:07.386446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:07.386288    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d1bd4748f35adc98495542aae3e22e67aa2833e9971c71b98b468786559a774"
Sep 09 03:21:10.378173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:10.378142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-s8dmr"
Sep 09 03:21:19.744583 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744554    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"]
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744845    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="extract"
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744856    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="extract"
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744865    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="util"
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744870    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="util"
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744883    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="pull"
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744887    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="pull"
Sep 09 03:21:19.744958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.744954    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a90c07f-c513-4881-8a59-e98f2fd9d687" containerName="extract"
Sep 09 03:21:19.747494 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.747478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.750030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.750008    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qsrqj\""
Sep 09 03:21:19.750030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.750019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 03:21:19.750214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.750019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 03:21:19.757123 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.757105    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"]
Sep 09 03:21:19.873421 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.873396    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7m44"]
Sep 09 03:21:19.890068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.890041    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7m44"]
Sep 09 03:21:19.890182 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.890122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:19.892391 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.892373    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6crhp\""
Sep 09 03:21:19.898150 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.898129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.898237 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.898156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl477\" (UniqueName: \"kubernetes.io/projected/7ea07851-e838-4e6b-abf2-15f6f30db954-kube-api-access-wl477\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.898237 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.898199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.999519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.999645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.999645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl477\" (UniqueName: \"kubernetes.io/projected/7ea07851-e838-4e6b-abf2-15f6f30db954-kube-api-access-wl477\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.999645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds569\" (UniqueName: \"kubernetes.io/projected/a169c3f9-d554-4677-a9a8-fe0b881d6da3-kube-api-access-ds569\") pod \"cert-manager-55b4d954d5-c7m44\" (UID: \"a169c3f9-d554-4677-a9a8-fe0b881d6da3\") " pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:19.999645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a169c3f9-d554-4677-a9a8-fe0b881d6da3-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7m44\" (UID: \"a169c3f9-d554-4677-a9a8-fe0b881d6da3\") " pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:19.999905 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:19.999957 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:19.999903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:20.017882 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.017860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl477\" (UniqueName: \"kubernetes.io/projected/7ea07851-e838-4e6b-abf2-15f6f30db954-kube-api-access-wl477\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:20.057509 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.057490    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:20.100401 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.100373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds569\" (UniqueName: \"kubernetes.io/projected/a169c3f9-d554-4677-a9a8-fe0b881d6da3-kube-api-access-ds569\") pod \"cert-manager-55b4d954d5-c7m44\" (UID: \"a169c3f9-d554-4677-a9a8-fe0b881d6da3\") " pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:20.100512 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.100420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a169c3f9-d554-4677-a9a8-fe0b881d6da3-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7m44\" (UID: \"a169c3f9-d554-4677-a9a8-fe0b881d6da3\") " pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:20.109018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.108995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a169c3f9-d554-4677-a9a8-fe0b881d6da3-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7m44\" (UID: \"a169c3f9-d554-4677-a9a8-fe0b881d6da3\") " pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:20.109194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.109028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds569\" (UniqueName: \"kubernetes.io/projected/a169c3f9-d554-4677-a9a8-fe0b881d6da3-kube-api-access-ds569\") pod \"cert-manager-55b4d954d5-c7m44\" (UID: \"a169c3f9-d554-4677-a9a8-fe0b881d6da3\") " pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:20.199378 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.199349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c7m44"
Sep 09 03:21:20.340787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.340763    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"]
Sep 09 03:21:20.342718 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:20.342692    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea07851_e838_4e6b_abf2_15f6f30db954.slice/crio-7cac7f534b16f9ac8768721ceb3bf5d58975c02eb12a4ada9a797dcd7fe12f91 WatchSource:0}: Error finding container 7cac7f534b16f9ac8768721ceb3bf5d58975c02eb12a4ada9a797dcd7fe12f91: Status 404 returned error can't find the container with id 7cac7f534b16f9ac8768721ceb3bf5d58975c02eb12a4ada9a797dcd7fe12f91
Sep 09 03:21:20.441703 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.441647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67" event={"ID":"7ea07851-e838-4e6b-abf2-15f6f30db954","Type":"ContainerStarted","Data":"a4f3e80d7a0a227222a0eed9b9785ff58e617e713aba0cd3a3ef95fb812184df"}
Sep 09 03:21:20.441825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.441721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67" event={"ID":"7ea07851-e838-4e6b-abf2-15f6f30db954","Type":"ContainerStarted","Data":"7cac7f534b16f9ac8768721ceb3bf5d58975c02eb12a4ada9a797dcd7fe12f91"}
Sep 09 03:21:20.485398 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:20.485372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7m44"]
Sep 09 03:21:20.515586 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:20.515532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda169c3f9_d554_4677_a9a8_fe0b881d6da3.slice/crio-473c0f7d86e9f96a53c29ef0498f5c7d3d2001e30c60fd5cf1d7f6df5635fd9e WatchSource:0}: Error finding container 473c0f7d86e9f96a53c29ef0498f5c7d3d2001e30c60fd5cf1d7f6df5635fd9e: Status 404 returned error can't find the container with id 473c0f7d86e9f96a53c29ef0498f5c7d3d2001e30c60fd5cf1d7f6df5635fd9e
Sep 09 03:21:21.446872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:21.446836    2580 generic.go:356] "Generic (PLEG): container finished" podID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerID="a4f3e80d7a0a227222a0eed9b9785ff58e617e713aba0cd3a3ef95fb812184df" exitCode=0
Sep 09 03:21:21.447303 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:21.446911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67" event={"ID":"7ea07851-e838-4e6b-abf2-15f6f30db954","Type":"ContainerDied","Data":"a4f3e80d7a0a227222a0eed9b9785ff58e617e713aba0cd3a3ef95fb812184df"}
Sep 09 03:21:21.448706 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:21.448645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c7m44" event={"ID":"a169c3f9-d554-4677-a9a8-fe0b881d6da3","Type":"ContainerStarted","Data":"75f6150e01722e570a5d762893bfc802ec1bb6b98f6319196a2bcdf3c38ff521"}
Sep 09 03:21:21.448787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:21.448707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c7m44" event={"ID":"a169c3f9-d554-4677-a9a8-fe0b881d6da3","Type":"ContainerStarted","Data":"473c0f7d86e9f96a53c29ef0498f5c7d3d2001e30c60fd5cf1d7f6df5635fd9e"}
Sep 09 03:21:21.485256 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:21.485205    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c7m44" podStartSLOduration=2.485190721 podStartE2EDuration="2.485190721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:21:21.483692797 +0000 UTC m=+492.183456755" watchObservedRunningTime="2026-09-09 03:21:21.485190721 +0000 UTC m=+492.184954683"
Sep 09 03:21:22.454610 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:22.454536    2580 generic.go:356] "Generic (PLEG): container finished" podID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerID="d16ec2a502fe33b3bd6174a6af20459cecd6dcf3ae9fd85547b9a0677083064a" exitCode=0
Sep 09 03:21:22.454995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:22.454625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67" event={"ID":"7ea07851-e838-4e6b-abf2-15f6f30db954","Type":"ContainerDied","Data":"d16ec2a502fe33b3bd6174a6af20459cecd6dcf3ae9fd85547b9a0677083064a"}
Sep 09 03:21:23.460942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:23.460906    2580 generic.go:356] "Generic (PLEG): container finished" podID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerID="d98deb464111908dde8747d72eef0815df4ae151c891633bec22342857858bf9" exitCode=0
Sep 09 03:21:23.461343 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:23.460987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67" event={"ID":"7ea07851-e838-4e6b-abf2-15f6f30db954","Type":"ContainerDied","Data":"d98deb464111908dde8747d72eef0815df4ae151c891633bec22342857858bf9"}
Sep 09 03:21:24.582280 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.582261    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:24.732139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.732071    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-util\") pod \"7ea07851-e838-4e6b-abf2-15f6f30db954\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") "
Sep 09 03:21:24.732139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.732113    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-bundle\") pod \"7ea07851-e838-4e6b-abf2-15f6f30db954\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") "
Sep 09 03:21:24.732316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.732190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl477\" (UniqueName: \"kubernetes.io/projected/7ea07851-e838-4e6b-abf2-15f6f30db954-kube-api-access-wl477\") pod \"7ea07851-e838-4e6b-abf2-15f6f30db954\" (UID: \"7ea07851-e838-4e6b-abf2-15f6f30db954\") "
Sep 09 03:21:24.732833 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.732799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-bundle" (OuterVolumeSpecName: "bundle") pod "7ea07851-e838-4e6b-abf2-15f6f30db954" (UID: "7ea07851-e838-4e6b-abf2-15f6f30db954"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:21:24.734352 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.734330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea07851-e838-4e6b-abf2-15f6f30db954-kube-api-access-wl477" (OuterVolumeSpecName: "kube-api-access-wl477") pod "7ea07851-e838-4e6b-abf2-15f6f30db954" (UID: "7ea07851-e838-4e6b-abf2-15f6f30db954"). InnerVolumeSpecName "kube-api-access-wl477". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:21:24.737897 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.737873    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-util" (OuterVolumeSpecName: "util") pod "7ea07851-e838-4e6b-abf2-15f6f30db954" (UID: "7ea07851-e838-4e6b-abf2-15f6f30db954"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:21:24.833474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.833443    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl477\" (UniqueName: \"kubernetes.io/projected/7ea07851-e838-4e6b-abf2-15f6f30db954-kube-api-access-wl477\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:24.833474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.833473    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:24.833474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:24.833483    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea07851-e838-4e6b-abf2-15f6f30db954-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:25.471972 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:25.471941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67" event={"ID":"7ea07851-e838-4e6b-abf2-15f6f30db954","Type":"ContainerDied","Data":"7cac7f534b16f9ac8768721ceb3bf5d58975c02eb12a4ada9a797dcd7fe12f91"}
Sep 09 03:21:25.471972 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:25.471959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmwm67"
Sep 09 03:21:25.472151 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:25.471990    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cac7f534b16f9ac8768721ceb3bf5d58975c02eb12a4ada9a797dcd7fe12f91"
Sep 09 03:21:35.996112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996071    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"]
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996394    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="pull"
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996406    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="pull"
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996427    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="extract"
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996433    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="extract"
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996439    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="util"
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996445    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="util"
Sep 09 03:21:35.996752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.996501    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea07851-e838-4e6b-abf2-15f6f30db954" containerName="extract"
Sep 09 03:21:35.998492 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:35.998476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.001161 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.001138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qsrqj\""
Sep 09 03:21:36.001317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.001293    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 03:21:36.002165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.002148    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 03:21:36.005189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.005165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.005274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.005194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.005274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.005213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r42w\" (UniqueName: \"kubernetes.io/projected/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-kube-api-access-4r42w\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.009777 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.009757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"]
Sep 09 03:21:36.106406 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.106376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.106546 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.106409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.106546 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.106438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r42w\" (UniqueName: \"kubernetes.io/projected/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-kube-api-access-4r42w\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.106819 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.106803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.106891 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.106869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.118713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.118692    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r42w\" (UniqueName: \"kubernetes.io/projected/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-kube-api-access-4r42w\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.307140 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.307087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:36.592613 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:36.592548    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"]
Sep 09 03:21:36.595824 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:36.595799    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5fdea2_c1e4_46c8_ab23_b9df64e4f2b5.slice/crio-1a8b3de1312a339039baab5e9280010a52249d3ddba68b395c0c26fecacbb18f WatchSource:0}: Error finding container 1a8b3de1312a339039baab5e9280010a52249d3ddba68b395c0c26fecacbb18f: Status 404 returned error can't find the container with id 1a8b3de1312a339039baab5e9280010a52249d3ddba68b395c0c26fecacbb18f
Sep 09 03:21:37.266738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.266705    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"]
Sep 09 03:21:37.277430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.277406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.280966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.280943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 03:21:37.281073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.281045    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 03:21:37.281258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.281231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jcqpg\""
Sep 09 03:21:37.281258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.281241    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 03:21:37.281949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.281933    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 03:21:37.288208 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.288189    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"]
Sep 09 03:21:37.318387 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.318359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1deafbda-c739-4427-9487-66d90e13aecc-manifests\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.318489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.318410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpzb8\" (UniqueName: \"kubernetes.io/projected/1deafbda-c739-4427-9487-66d90e13aecc-kube-api-access-wpzb8\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.318489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.318462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1deafbda-c739-4427-9487-66d90e13aecc-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.318605 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.318522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1deafbda-c739-4427-9487-66d90e13aecc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.318659 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.318589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1deafbda-c739-4427-9487-66d90e13aecc-tmp\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.419440 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.419385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1deafbda-c739-4427-9487-66d90e13aecc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.419440 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.419432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1deafbda-c739-4427-9487-66d90e13aecc-tmp\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.419604 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.419451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1deafbda-c739-4427-9487-66d90e13aecc-manifests\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.419604 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.419487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpzb8\" (UniqueName: \"kubernetes.io/projected/1deafbda-c739-4427-9487-66d90e13aecc-kube-api-access-wpzb8\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.419604 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.419518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1deafbda-c739-4427-9487-66d90e13aecc-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.419865 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.419845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1deafbda-c739-4427-9487-66d90e13aecc-manifests\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.421934 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.421909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1deafbda-c739-4427-9487-66d90e13aecc-tmp\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.422167 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.422143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1deafbda-c739-4427-9487-66d90e13aecc-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.422242 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.422185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1deafbda-c739-4427-9487-66d90e13aecc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.428063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.428042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpzb8\" (UniqueName: \"kubernetes.io/projected/1deafbda-c739-4427-9487-66d90e13aecc-kube-api-access-wpzb8\") pod \"opendatahub-operator-controller-manager-ff49c877-nnbch\" (UID: \"1deafbda-c739-4427-9487-66d90e13aecc\") " pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.519062 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.519036    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerID="8c74d91a9fd1143dc2ba67bbb1b3045737e196d7288f9ca6914a00ad1b8b122c" exitCode=0
Sep 09 03:21:37.519165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.519118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6" event={"ID":"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5","Type":"ContainerDied","Data":"8c74d91a9fd1143dc2ba67bbb1b3045737e196d7288f9ca6914a00ad1b8b122c"}
Sep 09 03:21:37.519218 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.519174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6" event={"ID":"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5","Type":"ContainerStarted","Data":"1a8b3de1312a339039baab5e9280010a52249d3ddba68b395c0c26fecacbb18f"}
Sep 09 03:21:37.588496 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.588476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:37.892249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:37.892222    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"]
Sep 09 03:21:37.893793 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:21:37.893768    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1deafbda_c739_4427_9487_66d90e13aecc.slice/crio-c42fe1128085d3f07c837cb114b4c38d9bba683fe93f5fac9bc24a7a53b23930 WatchSource:0}: Error finding container c42fe1128085d3f07c837cb114b4c38d9bba683fe93f5fac9bc24a7a53b23930: Status 404 returned error can't find the container with id c42fe1128085d3f07c837cb114b4c38d9bba683fe93f5fac9bc24a7a53b23930
Sep 09 03:21:38.528242 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:38.528207    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerID="4a9303514815fbc05b32a0567774a3bf2f870d273d1c7480d61196803bbb908c" exitCode=0
Sep 09 03:21:38.528710 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:38.528338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6" event={"ID":"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5","Type":"ContainerDied","Data":"4a9303514815fbc05b32a0567774a3bf2f870d273d1c7480d61196803bbb908c"}
Sep 09 03:21:38.530195 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:38.530169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch" event={"ID":"1deafbda-c739-4427-9487-66d90e13aecc","Type":"ContainerStarted","Data":"c42fe1128085d3f07c837cb114b4c38d9bba683fe93f5fac9bc24a7a53b23930"}
Sep 09 03:21:39.538359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:39.538325    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerID="58eb99d22ebc252a7f32c2c121bd2863a0612c62e87792e9e406033326e62d73" exitCode=0
Sep 09 03:21:39.538804 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:39.538410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6" event={"ID":"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5","Type":"ContainerDied","Data":"58eb99d22ebc252a7f32c2c121bd2863a0612c62e87792e9e406033326e62d73"}
Sep 09 03:21:40.757143 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.757123    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:40.844061 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.844040    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r42w\" (UniqueName: \"kubernetes.io/projected/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-kube-api-access-4r42w\") pod \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") "
Sep 09 03:21:40.844154 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.844072    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-bundle\") pod \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") "
Sep 09 03:21:40.844154 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.844128    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-util\") pod \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\" (UID: \"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5\") "
Sep 09 03:21:40.844807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.844784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-bundle" (OuterVolumeSpecName: "bundle") pod "bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" (UID: "bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:21:40.846102 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.846072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-kube-api-access-4r42w" (OuterVolumeSpecName: "kube-api-access-4r42w") pod "bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" (UID: "bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5"). InnerVolumeSpecName "kube-api-access-4r42w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:21:40.850185 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.850164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-util" (OuterVolumeSpecName: "util") pod "bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" (UID: "bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:21:40.945576 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.945555    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:40.945576 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.945575    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r42w\" (UniqueName: \"kubernetes.io/projected/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-kube-api-access-4r42w\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:40.945711 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:40.945584    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:21:41.556841 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:41.556810    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6"
Sep 09 03:21:41.556841 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:41.556822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xjfm6" event={"ID":"bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5","Type":"ContainerDied","Data":"1a8b3de1312a339039baab5e9280010a52249d3ddba68b395c0c26fecacbb18f"}
Sep 09 03:21:41.557053 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:41.556900    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a8b3de1312a339039baab5e9280010a52249d3ddba68b395c0c26fecacbb18f"
Sep 09 03:21:41.558596 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:41.558570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch" event={"ID":"1deafbda-c739-4427-9487-66d90e13aecc","Type":"ContainerStarted","Data":"225516e81fd16512df212e7501ba606b0a49869081bde32f27dd6215809ad86b"}
Sep 09 03:21:42.564554 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:42.564522    2580 generic.go:356] "Generic (PLEG): container finished" podID="1deafbda-c739-4427-9487-66d90e13aecc" containerID="225516e81fd16512df212e7501ba606b0a49869081bde32f27dd6215809ad86b" exitCode=0
Sep 09 03:21:42.564924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:42.564606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch" event={"ID":"1deafbda-c739-4427-9487-66d90e13aecc","Type":"ContainerDied","Data":"225516e81fd16512df212e7501ba606b0a49869081bde32f27dd6215809ad86b"}
Sep 09 03:21:43.571130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:43.571098    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch" event={"ID":"1deafbda-c739-4427-9487-66d90e13aecc","Type":"ContainerStarted","Data":"1f7995cefc3bfda06c9a219a39e9e8a14639d4fd7f6bcba16ba3def20ae14a66"}
Sep 09 03:21:43.571547 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:43.571235    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:21:43.595759 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:43.595710    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch" podStartSLOduration=2.37698144 podStartE2EDuration="6.595689839s" totalImagesPullingTime="3.214575308s" totalInitContainerRuntime="1.004133091s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:21:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:21:43.593944193 +0000 UTC m=+514.293708155" watchObservedRunningTime="2026-09-09 03:21:43.595689839 +0000 UTC m=+514.295453798"
Sep 09 03:21:54.577935 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:21:54.577906    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-ff49c877-nnbch"
Sep 09 03:22:07.871854 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.871779    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58f995d999-c62db"]
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872066    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="pull"
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872077    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="pull"
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872094    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="extract"
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872099    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="extract"
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872112    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="util"
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872117    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="util"
Sep 09 03:22:07.872298 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.872165    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5fdea2-c1e4-46c8-ab23-b9df64e4f2b5" containerName="extract"
Sep 09 03:22:07.874097 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.874069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:07.877591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.877567    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 03:22:07.877731 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.877567    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 03:22:07.877731 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.877567    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kwfhb\""
Sep 09 03:22:07.891495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.891470    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58f995d999-c62db"]
Sep 09 03:22:07.941812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.941791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/628ace00-06c2-4001-8132-018f8d6018a8-tmp\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:07.941927 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.941825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9rb9\" (UniqueName: \"kubernetes.io/projected/628ace00-06c2-4001-8132-018f8d6018a8-kube-api-access-d9rb9\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:07.941927 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:07.941860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ace00-06c2-4001-8132-018f8d6018a8-tls-certs\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.034603 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.034581    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rvt2k"]
Sep 09 03:22:08.036567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.036554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.038902 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.038887    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 03:22:08.038984 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.038929    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-94fpf\""
Sep 09 03:22:08.042327 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.042307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/628ace00-06c2-4001-8132-018f8d6018a8-tmp\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.042407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.042341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9rb9\" (UniqueName: \"kubernetes.io/projected/628ace00-06c2-4001-8132-018f8d6018a8-kube-api-access-d9rb9\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.042407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.042364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ace00-06c2-4001-8132-018f8d6018a8-tls-certs\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.045203 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.045186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/628ace00-06c2-4001-8132-018f8d6018a8-tmp\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.045515 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.045496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ace00-06c2-4001-8132-018f8d6018a8-tls-certs\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.053898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.053877    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rvt2k"]
Sep 09 03:22:08.061285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.061266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9rb9\" (UniqueName: \"kubernetes.io/projected/628ace00-06c2-4001-8132-018f8d6018a8-kube-api-access-d9rb9\") pod \"kube-auth-proxy-58f995d999-c62db\" (UID: \"628ace00-06c2-4001-8132-018f8d6018a8\") " pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.143569 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.143507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnknz\" (UniqueName: \"kubernetes.io/projected/a1176477-1719-48b3-b0b4-9e6a6020c315-kube-api-access-jnknz\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.143569 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.143565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1176477-1719-48b3-b0b4-9e6a6020c315-cert\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.183203 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.183176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db"
Sep 09 03:22:08.244400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.244366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1176477-1719-48b3-b0b4-9e6a6020c315-cert\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.244514 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.244459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnknz\" (UniqueName: \"kubernetes.io/projected/a1176477-1719-48b3-b0b4-9e6a6020c315-kube-api-access-jnknz\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.244573 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:08.244527    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 03:22:08.244622 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:08.244601    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1176477-1719-48b3-b0b4-9e6a6020c315-cert podName:a1176477-1719-48b3-b0b4-9e6a6020c315 nodeName:}" failed. No retries permitted until 2026-09-09 03:22:08.744577934 +0000 UTC m=+539.444341902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a1176477-1719-48b3-b0b4-9e6a6020c315-cert") pod "odh-model-controller-7998c67dc4-rvt2k" (UID: "a1176477-1719-48b3-b0b4-9e6a6020c315") : secret "odh-model-controller-webhook-cert" not found
Sep 09 03:22:08.254062 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.254044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnknz\" (UniqueName: \"kubernetes.io/projected/a1176477-1719-48b3-b0b4-9e6a6020c315-kube-api-access-jnknz\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.452749 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.452729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58f995d999-c62db"]
Sep 09 03:22:08.455400 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:08.455375    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628ace00_06c2_4001_8132_018f8d6018a8.slice/crio-8b37f0e3e1e4a44cd99dde1d1a340a088f8b7d1741f79b12a4e1425a791539f8 WatchSource:0}: Error finding container 8b37f0e3e1e4a44cd99dde1d1a340a088f8b7d1741f79b12a4e1425a791539f8: Status 404 returned error can't find the container with id 8b37f0e3e1e4a44cd99dde1d1a340a088f8b7d1741f79b12a4e1425a791539f8
Sep 09 03:22:08.682173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.682141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db" event={"ID":"628ace00-06c2-4001-8132-018f8d6018a8","Type":"ContainerStarted","Data":"8b37f0e3e1e4a44cd99dde1d1a340a088f8b7d1741f79b12a4e1425a791539f8"}
Sep 09 03:22:08.705384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.705310    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d9dh9"]
Sep 09 03:22:08.708180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.708160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:08.710922 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.710904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 03:22:08.720081 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.720047    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d9dh9"]
Sep 09 03:22:08.748200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.748171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5438cb-b06f-408f-8c7e-78269fdbafa6-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:08.748295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.748201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx6jx\" (UniqueName: \"kubernetes.io/projected/6f5438cb-b06f-408f-8c7e-78269fdbafa6-kube-api-access-sx6jx\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:08.748295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.748231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1176477-1719-48b3-b0b4-9e6a6020c315-cert\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.750643 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.750625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1176477-1719-48b3-b0b4-9e6a6020c315-cert\") pod \"odh-model-controller-7998c67dc4-rvt2k\" (UID: \"a1176477-1719-48b3-b0b4-9e6a6020c315\") " pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:08.849028 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.849001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5438cb-b06f-408f-8c7e-78269fdbafa6-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:08.849133 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.849032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx6jx\" (UniqueName: \"kubernetes.io/projected/6f5438cb-b06f-408f-8c7e-78269fdbafa6-kube-api-access-sx6jx\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:08.849174 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:08.849150    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 09 03:22:08.849217 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:08.849208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5438cb-b06f-408f-8c7e-78269fdbafa6-tls-certs podName:6f5438cb-b06f-408f-8c7e-78269fdbafa6 nodeName:}" failed. No retries permitted until 2026-09-09 03:22:09.349193138 +0000 UTC m=+540.048957078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6f5438cb-b06f-408f-8c7e-78269fdbafa6-tls-certs") pod "model-serving-api-54d89bd5fc-d9dh9" (UID: "6f5438cb-b06f-408f-8c7e-78269fdbafa6") : secret "model-serving-api-tls" not found
Sep 09 03:22:08.868238 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.868214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx6jx\" (UniqueName: \"kubernetes.io/projected/6f5438cb-b06f-408f-8c7e-78269fdbafa6-kube-api-access-sx6jx\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:08.946215 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:08.946177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:09.202227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.202195    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"]
Sep 09 03:22:09.205072 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.205046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.207445 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.207355    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 03:22:09.208850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.207938    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 03:22:09.208850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.208159    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 03:22:09.208850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.208427    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h2zjk\""
Sep 09 03:22:09.208850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.208698    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 03:22:09.223230 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.223098    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"]
Sep 09 03:22:09.225213 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.225007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rvt2k"]
Sep 09 03:22:09.227215 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:09.227092    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1176477_1719_48b3_b0b4_9e6a6020c315.slice/crio-66e5c3464edcdf6d1847fb4b92070f84a1a841f1781573dc697216ddd41bde5f WatchSource:0}: Error finding container 66e5c3464edcdf6d1847fb4b92070f84a1a841f1781573dc697216ddd41bde5f: Status 404 returned error can't find the container with id 66e5c3464edcdf6d1847fb4b92070f84a1a841f1781573dc697216ddd41bde5f
Sep 09 03:22:09.252902 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.252835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.252902 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.252870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.253070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.252900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.253070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.252945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.253070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.252969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krg2h\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-kube-api-access-krg2h\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.253070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.253027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.253070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.253062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1c18b2-2c46-498b-bc70-1e1846d75d61-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.353992 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.353950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5438cb-b06f-408f-8c7e-78269fdbafa6-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:09.354149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.354149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1c18b2-2c46-498b-bc70-1e1846d75d61-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.354149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.354149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.354364 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.354364 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.354364 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.354221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krg2h\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-kube-api-access-krg2h\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.355201 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.355171    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.357762 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.357664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5438cb-b06f-408f-8c7e-78269fdbafa6-tls-certs\") pod \"model-serving-api-54d89bd5fc-d9dh9\" (UID: \"6f5438cb-b06f-408f-8c7e-78269fdbafa6\") " pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:09.357956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.357914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1c18b2-2c46-498b-bc70-1e1846d75d61-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.358180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.358159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.359895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.359859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.360179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.360157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.363712 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.363655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krg2h\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-kube-api-access-krg2h\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.363987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.363966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nsqp8\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.519726 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.519635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:09.618557 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.618265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:09.698269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.698228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k" event={"ID":"a1176477-1719-48b3-b0b4-9e6a6020c315","Type":"ContainerStarted","Data":"66e5c3464edcdf6d1847fb4b92070f84a1a841f1781573dc697216ddd41bde5f"}
Sep 09 03:22:09.812052 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.812026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"]
Sep 09 03:22:09.909949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:09.909872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d9dh9"]
Sep 09 03:22:10.742651 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:10.742617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1c18b2_2c46_498b_bc70_1e1846d75d61.slice/crio-434845395a57a5e2c5ef6b6517b99a7c352b372dae6cf882adba321c33d18d42 WatchSource:0}: Error finding container 434845395a57a5e2c5ef6b6517b99a7c352b372dae6cf882adba321c33d18d42: Status 404 returned error can't find the container with id 434845395a57a5e2c5ef6b6517b99a7c352b372dae6cf882adba321c33d18d42
Sep 09 03:22:11.415757 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:11.415716    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5438cb_b06f_408f_8c7e_78269fdbafa6.slice/crio-b2aa1049ec4ed9c9311181b76fa2016fed62fcc21df2a946c0401d9ffc0d2b3a WatchSource:0}: Error finding container b2aa1049ec4ed9c9311181b76fa2016fed62fcc21df2a946c0401d9ffc0d2b3a: Status 404 returned error can't find the container with id b2aa1049ec4ed9c9311181b76fa2016fed62fcc21df2a946c0401d9ffc0d2b3a
Sep 09 03:22:11.715740 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:11.715708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9" event={"ID":"6f5438cb-b06f-408f-8c7e-78269fdbafa6","Type":"ContainerStarted","Data":"b2aa1049ec4ed9c9311181b76fa2016fed62fcc21df2a946c0401d9ffc0d2b3a"}
Sep 09 03:22:11.717317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:11.717280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" event={"ID":"2d1c18b2-2c46-498b-bc70-1e1846d75d61","Type":"ContainerStarted","Data":"434845395a57a5e2c5ef6b6517b99a7c352b372dae6cf882adba321c33d18d42"}
Sep 09 03:22:12.694926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:12.694899    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 03:22:13.731905 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:13.731868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db" event={"ID":"628ace00-06c2-4001-8132-018f8d6018a8","Type":"ContainerStarted","Data":"6a8ad387e5e6f15c539d49eab257ccba5142b3a8c790404660b365c4992db912"}
Sep 09 03:22:13.733755 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:13.733729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k" event={"ID":"a1176477-1719-48b3-b0b4-9e6a6020c315","Type":"ContainerStarted","Data":"a88e06212b1de5146ff7b8a70f826c86fe5c72ea862aa039d670e6ab3edafe87"}
Sep 09 03:22:13.733885 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:13.733798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:13.756046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:13.755990    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58f995d999-c62db" podStartSLOduration=2.521092427 podStartE2EDuration="6.75596929s" totalImagesPullingTime="4.234876863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:13.7534938 +0000 UTC m=+544.453257763" watchObservedRunningTime="2026-09-09 03:22:13.75596929 +0000 UTC m=+544.455733253"
Sep 09 03:22:13.776234 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:13.776179    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k" podStartSLOduration=2.280733524 podStartE2EDuration="5.776160884s" totalImagesPullingTime="3.49542736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:13.775802168 +0000 UTC m=+544.475566131" watchObservedRunningTime="2026-09-09 03:22:13.776160884 +0000 UTC m=+544.475924849"
Sep 09 03:22:14.747516 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:14.747470    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:22:14.747893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:14.747544    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:22:15.746646 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.746613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" event={"ID":"2d1c18b2-2c46-498b-bc70-1e1846d75d61","Type":"ContainerStarted","Data":"42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449"}
Sep 09 03:22:15.746872 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.746691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:15.748393 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.748358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9" event={"ID":"6f5438cb-b06f-408f-8c7e-78269fdbafa6","Type":"ContainerStarted","Data":"0e45b52c2a049e921f4b89e021f0813587f6dcf862ab542fa09b24ad0aaef522"}
Sep 09 03:22:15.748829 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.748506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:15.748829 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.748506    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-nsqp8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:22:15.748829 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.748545    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" podUID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:22:15.788410 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.788347    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" podStartSLOduration=2.786164891 podStartE2EDuration="6.788328507s" totalImagesPullingTime="4.002163616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:15.788092619 +0000 UTC m=+546.487856593" watchObservedRunningTime="2026-09-09 03:22:15.788328507 +0000 UTC m=+546.488092470"
Sep 09 03:22:15.815256 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:15.815215    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9" podStartSLOduration=4.482817321 podStartE2EDuration="7.815203708s" totalImagesPullingTime="3.332386387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:15.81394172 +0000 UTC m=+546.513705683" watchObservedRunningTime="2026-09-09 03:22:15.815203708 +0000 UTC m=+546.514967670"
Sep 09 03:22:16.272642 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.272614    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"]
Sep 09 03:22:16.274939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.274922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.279475 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.279454    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4gcn7\""
Sep 09 03:22:16.291105 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.291086    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"]
Sep 09 03:22:16.418420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418597 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418597 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418597 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418740 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf66s\" (UniqueName: \"kubernetes.io/projected/0493d4e1-8c33-4221-8152-a5d95f7eadc4-kube-api-access-pf66s\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418740 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418740 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.418740 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.418726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf66s\" (UniqueName: \"kubernetes.io/projected/0493d4e1-8c33-4221-8152-a5d95f7eadc4-kube-api-access-pf66s\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520426    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520955 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520871    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.520955 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.520896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.521147 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.521033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.522571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.522546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.522902 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.522855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.532786 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.532761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0493d4e1-8c33-4221-8152-a5d95f7eadc4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.532938 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.532922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf66s\" (UniqueName: \"kubernetes.io/projected/0493d4e1-8c33-4221-8152-a5d95f7eadc4-kube-api-access-pf66s\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp\" (UID: \"0493d4e1-8c33-4221-8152-a5d95f7eadc4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.585103 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.585081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:16.753577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.753555    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:16.866713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:16.866610    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"]
Sep 09 03:22:16.869575 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:16.869543    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0493d4e1_8c33_4221_8152_a5d95f7eadc4.slice/crio-5f30b36ac157893d9d6972128b676a2d7fc415ea31bf851f4a1778381f579712 WatchSource:0}: Error finding container 5f30b36ac157893d9d6972128b676a2d7fc415ea31bf851f4a1778381f579712: Status 404 returned error can't find the container with id 5f30b36ac157893d9d6972128b676a2d7fc415ea31bf851f4a1778381f579712
Sep 09 03:22:17.760117 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:17.760077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp" event={"ID":"0493d4e1-8c33-4221-8152-a5d95f7eadc4","Type":"ContainerStarted","Data":"5f30b36ac157893d9d6972128b676a2d7fc415ea31bf851f4a1778381f579712"}
Sep 09 03:22:18.443224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.443192    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"]
Sep 09 03:22:18.446725 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.446648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.449227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.449208    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 03:22:18.449906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.449890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cmz2l\""
Sep 09 03:22:18.459467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.459438    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"]
Sep 09 03:22:18.464373 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.464350    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"]
Sep 09 03:22:18.468191 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.468171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:18.472301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.471998    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t8rfn\""
Sep 09 03:22:18.472301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.472203    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 09 03:22:18.481040 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.481009    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"]
Sep 09 03:22:18.538620 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.538588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de5c32ed-0c73-4bd5-8adc-5abfe87296f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxd8h\" (UID: \"de5c32ed-0c73-4bd5-8adc-5abfe87296f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.538799 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.538627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b7bd951-d54a-4aa3-afc4-92647ef56aca-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:18.538799 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.538658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwl88\" (UniqueName: \"kubernetes.io/projected/de5c32ed-0c73-4bd5-8adc-5abfe87296f0-kube-api-access-cwl88\") pod \"kserve-controller-manager-d6f4b65b8-vxd8h\" (UID: \"de5c32ed-0c73-4bd5-8adc-5abfe87296f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.538799 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.538779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qc87\" (UniqueName: \"kubernetes.io/projected/1b7bd951-d54a-4aa3-afc4-92647ef56aca-kube-api-access-4qc87\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:18.639480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.639414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de5c32ed-0c73-4bd5-8adc-5abfe87296f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxd8h\" (UID: \"de5c32ed-0c73-4bd5-8adc-5abfe87296f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.639693 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.639501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b7bd951-d54a-4aa3-afc4-92647ef56aca-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:18.639693 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.639537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwl88\" (UniqueName: \"kubernetes.io/projected/de5c32ed-0c73-4bd5-8adc-5abfe87296f0-kube-api-access-cwl88\") pod \"kserve-controller-manager-d6f4b65b8-vxd8h\" (UID: \"de5c32ed-0c73-4bd5-8adc-5abfe87296f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.639693 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.639587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qc87\" (UniqueName: \"kubernetes.io/projected/1b7bd951-d54a-4aa3-afc4-92647ef56aca-kube-api-access-4qc87\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:18.639872 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:18.639701    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 09 03:22:18.639872 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:18.639775    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b7bd951-d54a-4aa3-afc4-92647ef56aca-cert podName:1b7bd951-d54a-4aa3-afc4-92647ef56aca nodeName:}" failed. No retries permitted until 2026-09-09 03:22:19.139754439 +0000 UTC m=+549.839518379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1b7bd951-d54a-4aa3-afc4-92647ef56aca-cert") pod "llmisvc-controller-manager-777c6bd7d5-nmsnb" (UID: "1b7bd951-d54a-4aa3-afc4-92647ef56aca") : secret "llmisvc-webhook-server-cert" not found
Sep 09 03:22:18.642534 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.642506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de5c32ed-0c73-4bd5-8adc-5abfe87296f0-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxd8h\" (UID: \"de5c32ed-0c73-4bd5-8adc-5abfe87296f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.651472 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.651446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwl88\" (UniqueName: \"kubernetes.io/projected/de5c32ed-0c73-4bd5-8adc-5abfe87296f0-kube-api-access-cwl88\") pod \"kserve-controller-manager-d6f4b65b8-vxd8h\" (UID: \"de5c32ed-0c73-4bd5-8adc-5abfe87296f0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:18.652159 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.652132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qc87\" (UniqueName: \"kubernetes.io/projected/1b7bd951-d54a-4aa3-afc4-92647ef56aca-kube-api-access-4qc87\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:18.760886 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:18.760855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:19.144954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.144870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b7bd951-d54a-4aa3-afc4-92647ef56aca-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:19.147751 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.147729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b7bd951-d54a-4aa3-afc4-92647ef56aca-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nmsnb\" (UID: \"1b7bd951-d54a-4aa3-afc4-92647ef56aca\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:19.322166 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.322112    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"]
Sep 09 03:22:19.322403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.322360    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" podUID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" containerName="discovery" containerID="cri-o://42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449" gracePeriod=30
Sep 09 03:22:19.379831 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.379778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:19.535086 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.534839    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:22:19.535086 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.534918    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:22:19.535086 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.534952    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:22:19.633432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.633413    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:19.737393 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.737310    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"]
Sep 09 03:22:19.740003 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:19.739973    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5c32ed_0c73_4bd5_8adc_5abfe87296f0.slice/crio-b9ebc7360175882fc130b420ea86edffc1596a3d9e0598a5606e27414bff7930 WatchSource:0}: Error finding container b9ebc7360175882fc130b420ea86edffc1596a3d9e0598a5606e27414bff7930: Status 404 returned error can't find the container with id b9ebc7360175882fc130b420ea86edffc1596a3d9e0598a5606e27414bff7930
Sep 09 03:22:19.740597 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.740570    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"]
Sep 09 03:22:19.741811 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:22:19.741595    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1b7bd951_d54a_4aa3_afc4_92647ef56aca.slice/crio-3b9b40f9eedc17dfa9d5aef1152712cf9d9ab579cd0283ceafce4e4f08ed86f4 WatchSource:0}: Error finding container 3b9b40f9eedc17dfa9d5aef1152712cf9d9ab579cd0283ceafce4e4f08ed86f4: Status 404 returned error can't find the container with id 3b9b40f9eedc17dfa9d5aef1152712cf9d9ab579cd0283ceafce4e4f08ed86f4
Sep 09 03:22:19.749886 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.749866    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-ca-configmap\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.749989 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.749914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-cacerts\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.749989 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.749935    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-dns-cert\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.749989 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.749963    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-token\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.750154 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.749991    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1c18b2-2c46-498b-bc70-1e1846d75d61-local-certs\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.750154 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.750034    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krg2h\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-kube-api-access-krg2h\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.750154 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.750111    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-kubeconfig\") pod \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\" (UID: \"2d1c18b2-2c46-498b-bc70-1e1846d75d61\") "
Sep 09 03:22:19.750301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.750279    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:22:19.750379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.750363    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-ca-configmap\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:19.752793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.752765    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-cacerts" (OuterVolumeSpecName: "cacerts") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:22:19.753236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.753192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:22:19.753455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.753410    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1c18b2-2c46-498b-bc70-1e1846d75d61-local-certs" (OuterVolumeSpecName: "local-certs") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:22:19.753455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.753433    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:22:19.753742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.753724    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-kube-api-access-krg2h" (OuterVolumeSpecName: "kube-api-access-krg2h") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "kube-api-access-krg2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:22:19.754923 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.754900    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "2d1c18b2-2c46-498b-bc70-1e1846d75d61" (UID: "2d1c18b2-2c46-498b-bc70-1e1846d75d61"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:22:19.772191 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.772167    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" containerID="42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449" exitCode=0
Sep 09 03:22:19.772558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.772210    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"
Sep 09 03:22:19.772558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.772259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" event={"ID":"2d1c18b2-2c46-498b-bc70-1e1846d75d61","Type":"ContainerDied","Data":"42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449"}
Sep 09 03:22:19.772558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.772314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8" event={"ID":"2d1c18b2-2c46-498b-bc70-1e1846d75d61","Type":"ContainerDied","Data":"434845395a57a5e2c5ef6b6517b99a7c352b372dae6cf882adba321c33d18d42"}
Sep 09 03:22:19.772558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.772344    2580 scope.go:117] "RemoveContainer" containerID="42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449"
Sep 09 03:22:19.773819 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.773795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h" event={"ID":"de5c32ed-0c73-4bd5-8adc-5abfe87296f0","Type":"ContainerStarted","Data":"b9ebc7360175882fc130b420ea86edffc1596a3d9e0598a5606e27414bff7930"}
Sep 09 03:22:19.775257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.775225    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb" event={"ID":"1b7bd951-d54a-4aa3-afc4-92647ef56aca","Type":"ContainerStarted","Data":"3b9b40f9eedc17dfa9d5aef1152712cf9d9ab579cd0283ceafce4e4f08ed86f4"}
Sep 09 03:22:19.777155 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.777131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp" event={"ID":"0493d4e1-8c33-4221-8152-a5d95f7eadc4","Type":"ContainerStarted","Data":"c10d83b1fb2fad7c1e8bbdf25a3ae9681dc711950b00fe152648c165831f03d0"}
Sep 09 03:22:19.782534 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.782515    2580 scope.go:117] "RemoveContainer" containerID="42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449"
Sep 09 03:22:19.782942 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:22:19.782913    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449\": container with ID starting with 42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449 not found: ID does not exist" containerID="42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449"
Sep 09 03:22:19.783023 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.782952    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449"} err="failed to get container status \"42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449\": rpc error: code = NotFound desc = could not find container \"42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449\": container with ID starting with 42b30e159b8a0df49f8ecf39ac355fd5e268521aba46706b4bda81146cc16449 not found: ID does not exist"
Sep 09 03:22:19.800457 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.800417    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp" podStartSLOduration=1.13822796 podStartE2EDuration="3.800398695s" totalImagesPullingTime="2.662170735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:19.798168969 +0000 UTC m=+550.497932930" watchObservedRunningTime="2026-09-09 03:22:19.800398695 +0000 UTC m=+550.500162661"
Sep 09 03:22:19.814521 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.814500    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"]
Sep 09 03:22:19.818463 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.818441    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nsqp8"]
Sep 09 03:22:19.851093 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.851072    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1c18b2-2c46-498b-bc70-1e1846d75d61-local-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:19.851204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.851096    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krg2h\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-kube-api-access-krg2h\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:19.851204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.851111    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-kubeconfig\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:19.851204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.851119    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-cacerts\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:19.851204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.851129    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-csr-dns-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:19.851204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:19.851137    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1c18b2-2c46-498b-bc70-1e1846d75d61-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:22:20.585901 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:20.585871    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:20.591262 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:20.591234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:20.786396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:20.786170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:20.787424 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:20.787398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp"
Sep 09 03:22:21.790812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:21.790779    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" path="/var/lib/kubelet/pods/2d1c18b2-2c46-498b-bc70-1e1846d75d61/volumes"
Sep 09 03:22:22.796918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:22.796882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h" event={"ID":"de5c32ed-0c73-4bd5-8adc-5abfe87296f0","Type":"ContainerStarted","Data":"0f63a861a478cc22b36cba476fb70387ddb4c5f6fab003c30862d0aadeb2ff35"}
Sep 09 03:22:22.797335 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:22.796984    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:22.798422 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:22.798401    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb" event={"ID":"1b7bd951-d54a-4aa3-afc4-92647ef56aca","Type":"ContainerStarted","Data":"0baa96405dfe619b9a69f5bb90f6b0de506f0a380720f4601e18518f2bf2627d"}
Sep 09 03:22:22.816241 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:22.816201    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h" podStartSLOduration=1.984104861 podStartE2EDuration="4.81618835s" totalImagesPullingTime="2.832083489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:22.813980093 +0000 UTC m=+553.513744055" watchObservedRunningTime="2026-09-09 03:22:22.81618835 +0000 UTC m=+553.515952311"
Sep 09 03:22:22.835604 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:22.835532    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb" podStartSLOduration=2.001586125 podStartE2EDuration="4.835520345s" totalImagesPullingTime="2.83393422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:22:22.835072773 +0000 UTC m=+553.534836728" watchObservedRunningTime="2026-09-09 03:22:22.835520345 +0000 UTC m=+553.535284307"
Sep 09 03:22:23.803202 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:23.803178    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:22:24.742539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:24.742507    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rvt2k"
Sep 09 03:22:26.757073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:26.757038    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-d9dh9"
Sep 09 03:22:53.808448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:53.808420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxd8h"
Sep 09 03:22:54.811217 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:22:54.811185    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nmsnb"
Sep 09 03:23:17.456343 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.456306    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"]
Sep 09 03:23:17.457589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.456651    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" containerName="discovery"
Sep 09 03:23:17.457589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.456662    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" containerName="discovery"
Sep 09 03:23:17.457589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.456758    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1c18b2-2c46-498b-bc70-1e1846d75d61" containerName="discovery"
Sep 09 03:23:17.458712 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.458697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.461273 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.461246    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qsrqj\""
Sep 09 03:23:17.462611 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.462593    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 03:23:17.462733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.462593    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 03:23:17.489226 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.489206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"]
Sep 09 03:23:17.581078 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.581053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.581207 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.581089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z86bq\" (UniqueName: \"kubernetes.io/projected/8f395077-d013-425f-9704-2cc7b4e1f739-kube-api-access-z86bq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.581207 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.581122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.581739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.581721    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"]
Sep 09 03:23:17.583954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.583940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.604899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.604878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"]
Sep 09 03:23:17.682242 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.682342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.682342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxdkw\" (UniqueName: \"kubernetes.io/projected/2f41eb21-3ea9-454b-8b71-6007bdac305a-kube-api-access-lxdkw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.682458 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.682458 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.682458 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z86bq\" (UniqueName: \"kubernetes.io/projected/8f395077-d013-425f-9704-2cc7b4e1f739-kube-api-access-z86bq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.682630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.682693 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.682652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.692146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.692119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z86bq\" (UniqueName: \"kubernetes.io/projected/8f395077-d013-425f-9704-2cc7b4e1f739-kube-api-access-z86bq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.699830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.699812    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"]
Sep 09 03:23:17.702123 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.702099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.728537 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.728517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"]
Sep 09 03:23:17.768040 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.768019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:17.782962 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.782937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.783051 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.782988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpc2f\" (UniqueName: \"kubernetes.io/projected/36b7a32f-f30a-4891-979d-59880b6d4e19-kube-api-access-kpc2f\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.783051 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.783020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.783180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.783064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.783180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.783173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.783284 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.783242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxdkw\" (UniqueName: \"kubernetes.io/projected/2f41eb21-3ea9-454b-8b71-6007bdac305a-kube-api-access-lxdkw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.783343 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.783326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.783465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.783435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.799516 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.799495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxdkw\" (UniqueName: \"kubernetes.io/projected/2f41eb21-3ea9-454b-8b71-6007bdac305a-kube-api-access-lxdkw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.884491 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.884467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpc2f\" (UniqueName: \"kubernetes.io/projected/36b7a32f-f30a-4891-979d-59880b6d4e19-kube-api-access-kpc2f\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.884625 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.884499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.884625 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.884519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.884859 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.884840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.884916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.884901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.885400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.885379    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"]
Sep 09 03:23:17.888801 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.888785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:17.892843 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.892827    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:17.914066 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.914041    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"]
Sep 09 03:23:17.914331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.914315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpc2f\" (UniqueName: \"kubernetes.io/projected/36b7a32f-f30a-4891-979d-59880b6d4e19-kube-api-access-kpc2f\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:17.985095 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.985063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:17.985221 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.985126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:17.985221 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:17.985196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gk45\" (UniqueName: \"kubernetes.io/projected/bae01735-851c-49e6-9f0d-1d75efe0f346-kube-api-access-5gk45\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.011000 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.010973    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:18.049100 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.049073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"]
Sep 09 03:23:18.050313 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:18.050289    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f395077_d013_425f_9704_2cc7b4e1f739.slice/crio-5463449ae75acd9e65f51782fe66a25fd902788aceb483aa36c1200fd55bcbf9 WatchSource:0}: Error finding container 5463449ae75acd9e65f51782fe66a25fd902788aceb483aa36c1200fd55bcbf9: Status 404 returned error can't find the container with id 5463449ae75acd9e65f51782fe66a25fd902788aceb483aa36c1200fd55bcbf9
Sep 09 03:23:18.085860 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.085828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gk45\" (UniqueName: \"kubernetes.io/projected/bae01735-851c-49e6-9f0d-1d75efe0f346-kube-api-access-5gk45\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.085975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.085920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.086037 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.085989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.086389 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.086367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.087833 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.087793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.099521 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.099501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gk45\" (UniqueName: \"kubernetes.io/projected/bae01735-851c-49e6-9f0d-1d75efe0f346-kube-api-access-5gk45\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.198792 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.194405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"]
Sep 09 03:23:18.198792 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.198511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:18.207064 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:18.207036    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f41eb21_3ea9_454b_8b71_6007bdac305a.slice/crio-36a886f7c6530dc02ed31f3a7183d89c90a4a939f1d58a9dc4c07e91cf83fd14 WatchSource:0}: Error finding container 36a886f7c6530dc02ed31f3a7183d89c90a4a939f1d58a9dc4c07e91cf83fd14: Status 404 returned error can't find the container with id 36a886f7c6530dc02ed31f3a7183d89c90a4a939f1d58a9dc4c07e91cf83fd14
Sep 09 03:23:18.312507 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.312416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"]
Sep 09 03:23:18.315314 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:18.315286    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b7a32f_f30a_4891_979d_59880b6d4e19.slice/crio-b6f482161802786281567576e2a1fd91e4cb448a83be9dbad306238658a1dc35 WatchSource:0}: Error finding container b6f482161802786281567576e2a1fd91e4cb448a83be9dbad306238658a1dc35: Status 404 returned error can't find the container with id b6f482161802786281567576e2a1fd91e4cb448a83be9dbad306238658a1dc35
Sep 09 03:23:18.489655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:18.489631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"]
Sep 09 03:23:18.491514 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:18.491490    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae01735_851c_49e6_9f0d_1d75efe0f346.slice/crio-417c3baae4b2e03a4697860210e8bfced3cf67a2513505b43da2e80eff5b1c21 WatchSource:0}: Error finding container 417c3baae4b2e03a4697860210e8bfced3cf67a2513505b43da2e80eff5b1c21: Status 404 returned error can't find the container with id 417c3baae4b2e03a4697860210e8bfced3cf67a2513505b43da2e80eff5b1c21
Sep 09 03:23:19.056480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.056447    2580 generic.go:356] "Generic (PLEG): container finished" podID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerID="a21c29128172b38d128c2ef2d51739cedd31e729e7586a24cf13cce5bf8c9648" exitCode=0
Sep 09 03:23:19.056661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.056538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s" event={"ID":"bae01735-851c-49e6-9f0d-1d75efe0f346","Type":"ContainerDied","Data":"a21c29128172b38d128c2ef2d51739cedd31e729e7586a24cf13cce5bf8c9648"}
Sep 09 03:23:19.056661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.056595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s" event={"ID":"bae01735-851c-49e6-9f0d-1d75efe0f346","Type":"ContainerStarted","Data":"417c3baae4b2e03a4697860210e8bfced3cf67a2513505b43da2e80eff5b1c21"}
Sep 09 03:23:19.057633 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.057618    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:23:19.058253 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.058229    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerID="2e4ac254e31d90a8d8252d3ab227f1270fb0bc12016e8199603d37adafb08872" exitCode=0
Sep 09 03:23:19.058348 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.058328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52" event={"ID":"2f41eb21-3ea9-454b-8b71-6007bdac305a","Type":"ContainerDied","Data":"2e4ac254e31d90a8d8252d3ab227f1270fb0bc12016e8199603d37adafb08872"}
Sep 09 03:23:19.058414 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.058373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52" event={"ID":"2f41eb21-3ea9-454b-8b71-6007bdac305a","Type":"ContainerStarted","Data":"36a886f7c6530dc02ed31f3a7183d89c90a4a939f1d58a9dc4c07e91cf83fd14"}
Sep 09 03:23:19.060000 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.059983    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f395077-d013-425f-9704-2cc7b4e1f739" containerID="a9e4676059baedf09614ba02a3a39b787bb64352bf1e54982be62fc26ad073c3" exitCode=0
Sep 09 03:23:19.060099 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.060063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v" event={"ID":"8f395077-d013-425f-9704-2cc7b4e1f739","Type":"ContainerDied","Data":"a9e4676059baedf09614ba02a3a39b787bb64352bf1e54982be62fc26ad073c3"}
Sep 09 03:23:19.060160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.060108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v" event={"ID":"8f395077-d013-425f-9704-2cc7b4e1f739","Type":"ContainerStarted","Data":"5463449ae75acd9e65f51782fe66a25fd902788aceb483aa36c1200fd55bcbf9"}
Sep 09 03:23:19.061809 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.061791    2580 generic.go:356] "Generic (PLEG): container finished" podID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerID="d9969a472190964b5fa9a20fbd74435db1022117eebd1a586ce688847d2f0afd" exitCode=0
Sep 09 03:23:19.061906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.061844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d" event={"ID":"36b7a32f-f30a-4891-979d-59880b6d4e19","Type":"ContainerDied","Data":"d9969a472190964b5fa9a20fbd74435db1022117eebd1a586ce688847d2f0afd"}
Sep 09 03:23:19.061906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:19.061872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d" event={"ID":"36b7a32f-f30a-4891-979d-59880b6d4e19","Type":"ContainerStarted","Data":"b6f482161802786281567576e2a1fd91e4cb448a83be9dbad306238658a1dc35"}
Sep 09 03:23:20.070252 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:20.070228    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerID="fa47e84c86e57f778e403a2cb887bbc2f26fd1f3ba96959b28cb769e87713256" exitCode=0
Sep 09 03:23:20.070573 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:20.070258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52" event={"ID":"2f41eb21-3ea9-454b-8b71-6007bdac305a","Type":"ContainerDied","Data":"fa47e84c86e57f778e403a2cb887bbc2f26fd1f3ba96959b28cb769e87713256"}
Sep 09 03:23:21.077653 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.077588    2580 generic.go:356] "Generic (PLEG): container finished" podID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerID="70de81512083d1a813f8364ab4d8d819fcab0d2030b646c6311a0607dbf41f18" exitCode=0
Sep 09 03:23:21.078046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.077689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s" event={"ID":"bae01735-851c-49e6-9f0d-1d75efe0f346","Type":"ContainerDied","Data":"70de81512083d1a813f8364ab4d8d819fcab0d2030b646c6311a0607dbf41f18"}
Sep 09 03:23:21.080062 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.080041    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerID="8b6d8987cafb59b3e0e0bc82e3a7bb8eaaee7e4f2c7bddcf4486139763f26ebb" exitCode=0
Sep 09 03:23:21.080167 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.080112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52" event={"ID":"2f41eb21-3ea9-454b-8b71-6007bdac305a","Type":"ContainerDied","Data":"8b6d8987cafb59b3e0e0bc82e3a7bb8eaaee7e4f2c7bddcf4486139763f26ebb"}
Sep 09 03:23:21.081837 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.081817    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f395077-d013-425f-9704-2cc7b4e1f739" containerID="015cde5c6b151d8d263b602281325c8c0c148f8b00e29987c2d62467352aa44a" exitCode=0
Sep 09 03:23:21.081935 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.081854    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v" event={"ID":"8f395077-d013-425f-9704-2cc7b4e1f739","Type":"ContainerDied","Data":"015cde5c6b151d8d263b602281325c8c0c148f8b00e29987c2d62467352aa44a"}
Sep 09 03:23:21.084199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.084179    2580 generic.go:356] "Generic (PLEG): container finished" podID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerID="4c7b4b2597ce244e477c2ea266461349b672c6b4ec1d918ca678b124cd713b3a" exitCode=0
Sep 09 03:23:21.084274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:21.084215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d" event={"ID":"36b7a32f-f30a-4891-979d-59880b6d4e19","Type":"ContainerDied","Data":"4c7b4b2597ce244e477c2ea266461349b672c6b4ec1d918ca678b124cd713b3a"}
Sep 09 03:23:21.283441 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:21.283413    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f395077_d013_425f_9704_2cc7b4e1f739.slice/crio-e94ae243203a1440f74a8cd9251914d83777dfecba19f47ff136872c473c5390.scope\": RecentStats: unable to find data in memory cache]"
Sep 09 03:23:22.090993 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.090922    2580 generic.go:356] "Generic (PLEG): container finished" podID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerID="3506dbed22786be28c6f50c495913960b817c19ecd97ed8a3c535f3e2f826a7c" exitCode=0
Sep 09 03:23:22.091331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.091009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s" event={"ID":"bae01735-851c-49e6-9f0d-1d75efe0f346","Type":"ContainerDied","Data":"3506dbed22786be28c6f50c495913960b817c19ecd97ed8a3c535f3e2f826a7c"}
Sep 09 03:23:22.093017 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.092994    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f395077-d013-425f-9704-2cc7b4e1f739" containerID="e94ae243203a1440f74a8cd9251914d83777dfecba19f47ff136872c473c5390" exitCode=0
Sep 09 03:23:22.093113 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.093067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v" event={"ID":"8f395077-d013-425f-9704-2cc7b4e1f739","Type":"ContainerDied","Data":"e94ae243203a1440f74a8cd9251914d83777dfecba19f47ff136872c473c5390"}
Sep 09 03:23:22.094876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.094854    2580 generic.go:356] "Generic (PLEG): container finished" podID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerID="61d96d4c59e8fb35664da1e742419f4518eb6479348031131280adda898d5c16" exitCode=0
Sep 09 03:23:22.094976 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.094933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d" event={"ID":"36b7a32f-f30a-4891-979d-59880b6d4e19","Type":"ContainerDied","Data":"61d96d4c59e8fb35664da1e742419f4518eb6479348031131280adda898d5c16"}
Sep 09 03:23:22.223594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.223569    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:22.315896 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.315873    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxdkw\" (UniqueName: \"kubernetes.io/projected/2f41eb21-3ea9-454b-8b71-6007bdac305a-kube-api-access-lxdkw\") pod \"2f41eb21-3ea9-454b-8b71-6007bdac305a\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") "
Sep 09 03:23:22.316015 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.315906    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-util\") pod \"2f41eb21-3ea9-454b-8b71-6007bdac305a\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") "
Sep 09 03:23:22.316015 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.315931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-bundle\") pod \"2f41eb21-3ea9-454b-8b71-6007bdac305a\" (UID: \"2f41eb21-3ea9-454b-8b71-6007bdac305a\") "
Sep 09 03:23:22.316481 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.316449    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-bundle" (OuterVolumeSpecName: "bundle") pod "2f41eb21-3ea9-454b-8b71-6007bdac305a" (UID: "2f41eb21-3ea9-454b-8b71-6007bdac305a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:22.318015 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.317992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f41eb21-3ea9-454b-8b71-6007bdac305a-kube-api-access-lxdkw" (OuterVolumeSpecName: "kube-api-access-lxdkw") pod "2f41eb21-3ea9-454b-8b71-6007bdac305a" (UID: "2f41eb21-3ea9-454b-8b71-6007bdac305a"). InnerVolumeSpecName "kube-api-access-lxdkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:23:22.321711 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.321681    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-util" (OuterVolumeSpecName: "util") pod "2f41eb21-3ea9-454b-8b71-6007bdac305a" (UID: "2f41eb21-3ea9-454b-8b71-6007bdac305a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:22.417379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.417327    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxdkw\" (UniqueName: \"kubernetes.io/projected/2f41eb21-3ea9-454b-8b71-6007bdac305a-kube-api-access-lxdkw\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:22.417379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.417349    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:22.417379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:22.417358    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f41eb21-3ea9-454b-8b71-6007bdac305a-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.101857 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.101827    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52"
Sep 09 03:23:23.102274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.101866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4clbq52" event={"ID":"2f41eb21-3ea9-454b-8b71-6007bdac305a","Type":"ContainerDied","Data":"36a886f7c6530dc02ed31f3a7183d89c90a4a939f1d58a9dc4c07e91cf83fd14"}
Sep 09 03:23:23.102274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.101907    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36a886f7c6530dc02ed31f3a7183d89c90a4a939f1d58a9dc4c07e91cf83fd14"
Sep 09 03:23:23.265687 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.265651    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:23.296121 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.296101    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:23.299545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.299525    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:23.323870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.323844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-util\") pod \"8f395077-d013-425f-9704-2cc7b4e1f739\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") "
Sep 09 03:23:23.323980 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.323921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z86bq\" (UniqueName: \"kubernetes.io/projected/8f395077-d013-425f-9704-2cc7b4e1f739-kube-api-access-z86bq\") pod \"8f395077-d013-425f-9704-2cc7b4e1f739\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") "
Sep 09 03:23:23.323980 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.323959    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-bundle\") pod \"8f395077-d013-425f-9704-2cc7b4e1f739\" (UID: \"8f395077-d013-425f-9704-2cc7b4e1f739\") "
Sep 09 03:23:23.324398 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.324375    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-bundle" (OuterVolumeSpecName: "bundle") pod "8f395077-d013-425f-9704-2cc7b4e1f739" (UID: "8f395077-d013-425f-9704-2cc7b4e1f739"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:23.326236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.326216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f395077-d013-425f-9704-2cc7b4e1f739-kube-api-access-z86bq" (OuterVolumeSpecName: "kube-api-access-z86bq") pod "8f395077-d013-425f-9704-2cc7b4e1f739" (UID: "8f395077-d013-425f-9704-2cc7b4e1f739"). InnerVolumeSpecName "kube-api-access-z86bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:23:23.345887 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.345857    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-util" (OuterVolumeSpecName: "util") pod "8f395077-d013-425f-9704-2cc7b4e1f739" (UID: "8f395077-d013-425f-9704-2cc7b4e1f739"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:23.424981 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.424923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gk45\" (UniqueName: \"kubernetes.io/projected/bae01735-851c-49e6-9f0d-1d75efe0f346-kube-api-access-5gk45\") pod \"bae01735-851c-49e6-9f0d-1d75efe0f346\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") "
Sep 09 03:23:23.425108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.424988    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpc2f\" (UniqueName: \"kubernetes.io/projected/36b7a32f-f30a-4891-979d-59880b6d4e19-kube-api-access-kpc2f\") pod \"36b7a32f-f30a-4891-979d-59880b6d4e19\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") "
Sep 09 03:23:23.425108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-bundle\") pod \"bae01735-851c-49e6-9f0d-1d75efe0f346\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") "
Sep 09 03:23:23.425108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425028    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-util\") pod \"bae01735-851c-49e6-9f0d-1d75efe0f346\" (UID: \"bae01735-851c-49e6-9f0d-1d75efe0f346\") "
Sep 09 03:23:23.425264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-util\") pod \"36b7a32f-f30a-4891-979d-59880b6d4e19\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") "
Sep 09 03:23:23.425264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-bundle\") pod \"36b7a32f-f30a-4891-979d-59880b6d4e19\" (UID: \"36b7a32f-f30a-4891-979d-59880b6d4e19\") "
Sep 09 03:23:23.425466 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425445    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z86bq\" (UniqueName: \"kubernetes.io/projected/8f395077-d013-425f-9704-2cc7b4e1f739-kube-api-access-z86bq\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.425524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425473    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.425524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425487    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f395077-d013-425f-9704-2cc7b4e1f739-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.425644 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-bundle" (OuterVolumeSpecName: "bundle") pod "bae01735-851c-49e6-9f0d-1d75efe0f346" (UID: "bae01735-851c-49e6-9f0d-1d75efe0f346"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:23.425980 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.425958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-bundle" (OuterVolumeSpecName: "bundle") pod "36b7a32f-f30a-4891-979d-59880b6d4e19" (UID: "36b7a32f-f30a-4891-979d-59880b6d4e19"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:23.427283 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.427255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae01735-851c-49e6-9f0d-1d75efe0f346-kube-api-access-5gk45" (OuterVolumeSpecName: "kube-api-access-5gk45") pod "bae01735-851c-49e6-9f0d-1d75efe0f346" (UID: "bae01735-851c-49e6-9f0d-1d75efe0f346"). InnerVolumeSpecName "kube-api-access-5gk45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:23:23.427283 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.427279    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b7a32f-f30a-4891-979d-59880b6d4e19-kube-api-access-kpc2f" (OuterVolumeSpecName: "kube-api-access-kpc2f") pod "36b7a32f-f30a-4891-979d-59880b6d4e19" (UID: "36b7a32f-f30a-4891-979d-59880b6d4e19"). InnerVolumeSpecName "kube-api-access-kpc2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:23:23.434276 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.434253    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-util" (OuterVolumeSpecName: "util") pod "bae01735-851c-49e6-9f0d-1d75efe0f346" (UID: "bae01735-851c-49e6-9f0d-1d75efe0f346"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:23.434493 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.434476    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-util" (OuterVolumeSpecName: "util") pod "36b7a32f-f30a-4891-979d-59880b6d4e19" (UID: "36b7a32f-f30a-4891-979d-59880b6d4e19"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:23.526263 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.526231    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpc2f\" (UniqueName: \"kubernetes.io/projected/36b7a32f-f30a-4891-979d-59880b6d4e19-kube-api-access-kpc2f\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.526263 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.526263    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.526263 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.526272    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bae01735-851c-49e6-9f0d-1d75efe0f346-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.526419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.526280    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-util\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.526419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.526287    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36b7a32f-f30a-4891-979d-59880b6d4e19-bundle\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:23.526419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:23.526294    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gk45\" (UniqueName: \"kubernetes.io/projected/bae01735-851c-49e6-9f0d-1d75efe0f346-kube-api-access-5gk45\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:24.109063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.108997    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v"
Sep 09 03:23:24.109063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.109005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62p26v" event={"ID":"8f395077-d013-425f-9704-2cc7b4e1f739","Type":"ContainerDied","Data":"5463449ae75acd9e65f51782fe66a25fd902788aceb483aa36c1200fd55bcbf9"}
Sep 09 03:23:24.109063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.109053    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5463449ae75acd9e65f51782fe66a25fd902788aceb483aa36c1200fd55bcbf9"
Sep 09 03:23:24.111137 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.111121    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d"
Sep 09 03:23:24.111259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.111179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amz56d" event={"ID":"36b7a32f-f30a-4891-979d-59880b6d4e19","Type":"ContainerDied","Data":"b6f482161802786281567576e2a1fd91e4cb448a83be9dbad306238658a1dc35"}
Sep 09 03:23:24.111259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.111215    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6f482161802786281567576e2a1fd91e4cb448a83be9dbad306238658a1dc35"
Sep 09 03:23:24.113555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.113537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s" event={"ID":"bae01735-851c-49e6-9f0d-1d75efe0f346","Type":"ContainerDied","Data":"417c3baae4b2e03a4697860210e8bfced3cf67a2513505b43da2e80eff5b1c21"}
Sep 09 03:23:24.113645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.113566    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gj74s"
Sep 09 03:23:24.113645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:24.113566    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="417c3baae4b2e03a4697860210e8bfced3cf67a2513505b43da2e80eff5b1c21"
Sep 09 03:23:36.646974 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.646900    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"]
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647216    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647237    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647243    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647252    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647259    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647267    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647272    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647277    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647282    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647289    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647293    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647301    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647306    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647316    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647320    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647327    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647332    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647340    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647344    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="extract"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647354    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647358    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="util"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647364    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="pull"
Sep 09 03:23:36.647384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647369    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="pull"
Sep 09 03:23:36.648430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647423    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f395077-d013-425f-9704-2cc7b4e1f739" containerName="extract"
Sep 09 03:23:36.648430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647431    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f41eb21-3ea9-454b-8b71-6007bdac305a" containerName="extract"
Sep 09 03:23:36.648430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647437    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b7a32f-f30a-4891-979d-59880b6d4e19" containerName="extract"
Sep 09 03:23:36.648430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.647444    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bae01735-851c-49e6-9f0d-1d75efe0f346" containerName="extract"
Sep 09 03:23:36.650353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.650339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.653496 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.653472    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 03:23:36.653630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.653582    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 03:23:36.653630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.653608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gdnld\""
Sep 09 03:23:36.686397 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.686376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"]
Sep 09 03:23:36.727565 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.727540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqvdj\" (UniqueName: \"kubernetes.io/projected/c94a0938-2d81-4e30-9bea-31155074eab8-kube-api-access-tqvdj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.727664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.727589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c94a0938-2d81-4e30-9bea-31155074eab8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.828146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.828124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c94a0938-2d81-4e30-9bea-31155074eab8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.828238 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.828210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqvdj\" (UniqueName: \"kubernetes.io/projected/c94a0938-2d81-4e30-9bea-31155074eab8-kube-api-access-tqvdj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.828543 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.828525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c94a0938-2d81-4e30-9bea-31155074eab8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.859276 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.859251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqvdj\" (UniqueName: \"kubernetes.io/projected/c94a0938-2d81-4e30-9bea-31155074eab8-kube-api-access-tqvdj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:36.960328 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:36.960255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:37.267316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:37.267291    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"]
Sep 09 03:23:37.269227 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:37.269203    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc94a0938_2d81_4e30_9bea_31155074eab8.slice/crio-bc10b13fb2e02e3b7851976d3f47f69ad3692c3fe01c31879f3d86619c40f359 WatchSource:0}: Error finding container bc10b13fb2e02e3b7851976d3f47f69ad3692c3fe01c31879f3d86619c40f359: Status 404 returned error can't find the container with id bc10b13fb2e02e3b7851976d3f47f69ad3692c3fe01c31879f3d86619c40f359
Sep 09 03:23:38.195064 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.195018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg" event={"ID":"c94a0938-2d81-4e30-9bea-31155074eab8","Type":"ContainerStarted","Data":"bc10b13fb2e02e3b7851976d3f47f69ad3692c3fe01c31879f3d86619c40f359"}
Sep 09 03:23:38.567890 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.567856    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"]
Sep 09 03:23:38.571639 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.571619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:38.577360 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.577338    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-566gc\""
Sep 09 03:23:38.587405 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.587380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"]
Sep 09 03:23:38.641897 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.641857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktgxn\" (UniqueName: \"kubernetes.io/projected/90014e78-9e10-46a8-9ee2-23b65e204916-kube-api-access-ktgxn\") pod \"limitador-operator-controller-manager-74495c69f8-pww7s\" (UID: \"90014e78-9e10-46a8-9ee2-23b65e204916\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:38.743420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.743383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktgxn\" (UniqueName: \"kubernetes.io/projected/90014e78-9e10-46a8-9ee2-23b65e204916-kube-api-access-ktgxn\") pod \"limitador-operator-controller-manager-74495c69f8-pww7s\" (UID: \"90014e78-9e10-46a8-9ee2-23b65e204916\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:38.764635 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.764609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktgxn\" (UniqueName: \"kubernetes.io/projected/90014e78-9e10-46a8-9ee2-23b65e204916-kube-api-access-ktgxn\") pod \"limitador-operator-controller-manager-74495c69f8-pww7s\" (UID: \"90014e78-9e10-46a8-9ee2-23b65e204916\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:38.887123 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:38.887057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:39.957610 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:39.957584    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"]
Sep 09 03:23:39.958876 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:39.958843    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90014e78_9e10_46a8_9ee2_23b65e204916.slice/crio-630e07e5f0c4a8e934501d7b5d73f3e1531d4789da156ef092c3fdcfae5951fa WatchSource:0}: Error finding container 630e07e5f0c4a8e934501d7b5d73f3e1531d4789da156ef092c3fdcfae5951fa: Status 404 returned error can't find the container with id 630e07e5f0c4a8e934501d7b5d73f3e1531d4789da156ef092c3fdcfae5951fa
Sep 09 03:23:40.208882 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:40.208810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s" event={"ID":"90014e78-9e10-46a8-9ee2-23b65e204916","Type":"ContainerStarted","Data":"630e07e5f0c4a8e934501d7b5d73f3e1531d4789da156ef092c3fdcfae5951fa"}
Sep 09 03:23:41.136390 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.136363    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"]
Sep 09 03:23:41.140237 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.140216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:41.148934 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.148910    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-f898r\""
Sep 09 03:23:41.149189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.149169    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 03:23:41.164017 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.163992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"]
Sep 09 03:23:41.267140 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.267054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b9k7\" (UniqueName: \"kubernetes.io/projected/dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2-kube-api-access-6b9k7\") pod \"dns-operator-controller-manager-7b84cd7c9c-7clx8\" (UID: \"dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:41.367594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.367563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b9k7\" (UniqueName: \"kubernetes.io/projected/dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2-kube-api-access-6b9k7\") pod \"dns-operator-controller-manager-7b84cd7c9c-7clx8\" (UID: \"dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:41.393809 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.393750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b9k7\" (UniqueName: \"kubernetes.io/projected/dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2-kube-api-access-6b9k7\") pod \"dns-operator-controller-manager-7b84cd7c9c-7clx8\" (UID: \"dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:41.453870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.453846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:41.773462 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:41.773432    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc7d2ce_165c_4f71_b5bf_3437dc6a6be2.slice/crio-8a03c71ac356ba632dc68ddc2b1a38947a249ec7b98b5149123f745ea64a1520 WatchSource:0}: Error finding container 8a03c71ac356ba632dc68ddc2b1a38947a249ec7b98b5149123f745ea64a1520: Status 404 returned error can't find the container with id 8a03c71ac356ba632dc68ddc2b1a38947a249ec7b98b5149123f745ea64a1520
Sep 09 03:23:41.778453 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:41.778424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"]
Sep 09 03:23:42.221940 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:42.221912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg" event={"ID":"c94a0938-2d81-4e30-9bea-31155074eab8","Type":"ContainerStarted","Data":"4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c"}
Sep 09 03:23:42.223090 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:42.223069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:42.223367 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:42.223350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8" event={"ID":"dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2","Type":"ContainerStarted","Data":"8a03c71ac356ba632dc68ddc2b1a38947a249ec7b98b5149123f745ea64a1520"}
Sep 09 03:23:42.311100 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:42.311053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg" podStartSLOduration=2.185535956 podStartE2EDuration="6.311038101s" totalImagesPullingTime="4.125502145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:23:42.309546933 +0000 UTC m=+633.009310894" watchObservedRunningTime="2026-09-09 03:23:42.311038101 +0000 UTC m=+633.010802063"
Sep 09 03:23:43.233199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:43.233164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s" event={"ID":"90014e78-9e10-46a8-9ee2-23b65e204916","Type":"ContainerStarted","Data":"3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a"}
Sep 09 03:23:43.270181 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:43.270135    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s" podStartSLOduration=3.013296588 podStartE2EDuration="5.27011193s" totalImagesPullingTime="2.256815342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:23:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:23:43.269796978 +0000 UTC m=+633.969560939" watchObservedRunningTime="2026-09-09 03:23:43.27011193 +0000 UTC m=+633.969875891"
Sep 09 03:23:44.240891 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.240857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8" event={"ID":"dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2","Type":"ContainerStarted","Data":"ac33e6a22940183a06f8618f57e82502cba15a97a4c08c6bbbcb435007b2e9ab"}
Sep 09 03:23:44.241261 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.241045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:44.241261 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.241247    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:44.284111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.284066    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8" podStartSLOduration=1.340259728 podStartE2EDuration="3.284053228s" totalImagesPullingTime="1.9437935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:23:44.282374746 +0000 UTC m=+634.982138708" watchObservedRunningTime="2026-09-09 03:23:44.284053228 +0000 UTC m=+634.983817189"
Sep 09 03:23:44.550329 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.550258    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-lppql"]
Sep 09 03:23:44.553765 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.553746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:44.557480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.557461    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4bq65\""
Sep 09 03:23:44.578362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.578343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-lppql"]
Sep 09 03:23:44.696262 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.696237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bbrg\" (UniqueName: \"kubernetes.io/projected/15e88706-5650-4175-aca6-21a55be6e0eb-kube-api-access-7bbrg\") pod \"authorino-operator-76b478bd78-lppql\" (UID: \"15e88706-5650-4175-aca6-21a55be6e0eb\") " pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:44.797139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.797115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bbrg\" (UniqueName: \"kubernetes.io/projected/15e88706-5650-4175-aca6-21a55be6e0eb-kube-api-access-7bbrg\") pod \"authorino-operator-76b478bd78-lppql\" (UID: \"15e88706-5650-4175-aca6-21a55be6e0eb\") " pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:44.812752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.812691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bbrg\" (UniqueName: \"kubernetes.io/projected/15e88706-5650-4175-aca6-21a55be6e0eb-kube-api-access-7bbrg\") pod \"authorino-operator-76b478bd78-lppql\" (UID: \"15e88706-5650-4175-aca6-21a55be6e0eb\") " pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:44.863325 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:44.863299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:45.162791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:45.162763    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-lppql"]
Sep 09 03:23:45.165531 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:45.165503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e88706_5650_4175_aca6_21a55be6e0eb.slice/crio-6a3c31ecca3c1e9a39002c8686e9bce2dea42ce19e21428884fa7d65261e50ec WatchSource:0}: Error finding container 6a3c31ecca3c1e9a39002c8686e9bce2dea42ce19e21428884fa7d65261e50ec: Status 404 returned error can't find the container with id 6a3c31ecca3c1e9a39002c8686e9bce2dea42ce19e21428884fa7d65261e50ec
Sep 09 03:23:45.247914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:45.247889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-lppql" event={"ID":"15e88706-5650-4175-aca6-21a55be6e0eb","Type":"ContainerStarted","Data":"6a3c31ecca3c1e9a39002c8686e9bce2dea42ce19e21428884fa7d65261e50ec"}
Sep 09 03:23:48.265573 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:48.265538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-lppql" event={"ID":"15e88706-5650-4175-aca6-21a55be6e0eb","Type":"ContainerStarted","Data":"b5464735e0661273073db1a95ad1e0e6c971b8b997cd04fde035d338ffbb4afc"}
Sep 09 03:23:48.265962 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:48.265618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:48.317498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:48.317452    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-lppql" podStartSLOduration=1.923560624 podStartE2EDuration="4.317437965s" totalImagesPullingTime="2.393877341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:23:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:23:48.314482846 +0000 UTC m=+639.014246807" watchObservedRunningTime="2026-09-09 03:23:48.317437965 +0000 UTC m=+639.017201927"
Sep 09 03:23:53.235733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:53.235707    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:55.250006 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:55.249975    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7clx8"
Sep 09 03:23:55.250428 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:55.250085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:56.265169 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.265139    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"]
Sep 09 03:23:56.265545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.265299    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" containerName="manager" containerID="cri-o://4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c" gracePeriod=2
Sep 09 03:23:56.281236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.281211    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"]
Sep 09 03:23:56.293874 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.293838    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c94a0938-2d81-4e30-9bea-31155074eab8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\": pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:56.301714 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.301691    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"]
Sep 09 03:23:56.301871 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.301851    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" containerName="manager" containerID="cri-o://3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a" gracePeriod=2
Sep 09 03:23:56.304844 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.304812    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:56.315453 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.315432    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"]
Sep 09 03:23:56.365741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.365719    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"]
Sep 09 03:23:56.366286 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.366270    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" containerName="manager"
Sep 09 03:23:56.366359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.366289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" containerName="manager"
Sep 09 03:23:56.366359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.366313    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" containerName="manager"
Sep 09 03:23:56.366359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.366318    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" containerName="manager"
Sep 09 03:23:56.366454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.366396    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" containerName="manager"
Sep 09 03:23:56.366454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.366404    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" containerName="manager"
Sep 09 03:23:56.382013 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.381987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.392703 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.392655    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pww7s\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:56.403165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.403144    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"]
Sep 09 03:23:56.406424 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.406403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:23:56.425333 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.425296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"]
Sep 09 03:23:56.478432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.478399    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"]
Sep 09 03:23:56.492781 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.492751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq54f\" (UniqueName: \"kubernetes.io/projected/9f54ee7d-9762-4394-a581-df98e2fed938-kube-api-access-nq54f\") pod \"kuadrant-operator-controller-manager-76f8f686bf-59fd7\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.492962 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.492842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f54ee7d-9762-4394-a581-df98e2fed938-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-59fd7\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.569438 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.569407    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:56.574470 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.574442    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:56.575191 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.575172    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:56.578162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.578145    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:56.593791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.593774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq54f\" (UniqueName: \"kubernetes.io/projected/9f54ee7d-9762-4394-a581-df98e2fed938-kube-api-access-nq54f\") pod \"kuadrant-operator-controller-manager-76f8f686bf-59fd7\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.593876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.593816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f54ee7d-9762-4394-a581-df98e2fed938-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-59fd7\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.593876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.593852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv7zb\" (UniqueName: \"kubernetes.io/projected/5e75a48c-96cf-431e-a095-b0de7adc0039-kube-api-access-dv7zb\") pod \"limitador-operator-controller-manager-74495c69f8-bxfvm\" (UID: \"5e75a48c-96cf-431e-a095-b0de7adc0039\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:23:56.594120 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.594094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f54ee7d-9762-4394-a581-df98e2fed938-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-59fd7\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.634660 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.634636    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pww7s\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:56.637912 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.637894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq54f\" (UniqueName: \"kubernetes.io/projected/9f54ee7d-9762-4394-a581-df98e2fed938-kube-api-access-nq54f\") pod \"kuadrant-operator-controller-manager-76f8f686bf-59fd7\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.638968 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.638948    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pww7s\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:56.641191 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:56.641173    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:56.694689 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.694653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktgxn\" (UniqueName: \"kubernetes.io/projected/90014e78-9e10-46a8-9ee2-23b65e204916-kube-api-access-ktgxn\") pod \"90014e78-9e10-46a8-9ee2-23b65e204916\" (UID: \"90014e78-9e10-46a8-9ee2-23b65e204916\") "
Sep 09 03:23:56.694785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.694772    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqvdj\" (UniqueName: \"kubernetes.io/projected/c94a0938-2d81-4e30-9bea-31155074eab8-kube-api-access-tqvdj\") pod \"c94a0938-2d81-4e30-9bea-31155074eab8\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") "
Sep 09 03:23:56.694832 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.694821    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c94a0938-2d81-4e30-9bea-31155074eab8-extensions-socket-volume\") pod \"c94a0938-2d81-4e30-9bea-31155074eab8\" (UID: \"c94a0938-2d81-4e30-9bea-31155074eab8\") "
Sep 09 03:23:56.694938 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.694923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv7zb\" (UniqueName: \"kubernetes.io/projected/5e75a48c-96cf-431e-a095-b0de7adc0039-kube-api-access-dv7zb\") pod \"limitador-operator-controller-manager-74495c69f8-bxfvm\" (UID: \"5e75a48c-96cf-431e-a095-b0de7adc0039\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:23:56.695278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.695245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c94a0938-2d81-4e30-9bea-31155074eab8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c94a0938-2d81-4e30-9bea-31155074eab8" (UID: "c94a0938-2d81-4e30-9bea-31155074eab8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:23:56.696825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.696804    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c94a0938-2d81-4e30-9bea-31155074eab8-kube-api-access-tqvdj" (OuterVolumeSpecName: "kube-api-access-tqvdj") pod "c94a0938-2d81-4e30-9bea-31155074eab8" (UID: "c94a0938-2d81-4e30-9bea-31155074eab8"). InnerVolumeSpecName "kube-api-access-tqvdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:23:56.696825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.696811    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90014e78-9e10-46a8-9ee2-23b65e204916-kube-api-access-ktgxn" (OuterVolumeSpecName: "kube-api-access-ktgxn") pod "90014e78-9e10-46a8-9ee2-23b65e204916" (UID: "90014e78-9e10-46a8-9ee2-23b65e204916"). InnerVolumeSpecName "kube-api-access-ktgxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:23:56.713448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.713428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:56.736964 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.736936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv7zb\" (UniqueName: \"kubernetes.io/projected/5e75a48c-96cf-431e-a095-b0de7adc0039-kube-api-access-dv7zb\") pod \"limitador-operator-controller-manager-74495c69f8-bxfvm\" (UID: \"5e75a48c-96cf-431e-a095-b0de7adc0039\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:23:56.796344 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.796317    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktgxn\" (UniqueName: \"kubernetes.io/projected/90014e78-9e10-46a8-9ee2-23b65e204916-kube-api-access-ktgxn\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:56.796344 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.796342    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqvdj\" (UniqueName: \"kubernetes.io/projected/c94a0938-2d81-4e30-9bea-31155074eab8-kube-api-access-tqvdj\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:56.796527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:56.796352    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c94a0938-2d81-4e30-9bea-31155074eab8-extensions-socket-volume\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:23:57.021734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.021710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:23:57.021976 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:57.021952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f54ee7d_9762_4394_a581_df98e2fed938.slice/crio-45d4daa32ab170919c483705e280d92db5f07cb2608d193f7d9e998986c422c6 WatchSource:0}: Error finding container 45d4daa32ab170919c483705e280d92db5f07cb2608d193f7d9e998986c422c6: Status 404 returned error can't find the container with id 45d4daa32ab170919c483705e280d92db5f07cb2608d193f7d9e998986c422c6
Sep 09 03:23:57.026531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.026509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"]
Sep 09 03:23:57.316531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.316501    2580 generic.go:356] "Generic (PLEG): container finished" podID="c94a0938-2d81-4e30-9bea-31155074eab8" containerID="4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c" exitCode=0
Sep 09 03:23:57.316909 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.316553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:57.316909 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.316612    2580 scope.go:117] "RemoveContainer" containerID="4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c"
Sep 09 03:23:57.318068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.318051    2580 generic.go:356] "Generic (PLEG): container finished" podID="90014e78-9e10-46a8-9ee2-23b65e204916" containerID="3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a" exitCode=0
Sep 09 03:23:57.318168 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.318095    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:57.320305 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.320277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7" event={"ID":"9f54ee7d-9762-4394-a581-df98e2fed938","Type":"ContainerStarted","Data":"6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5"}
Sep 09 03:23:57.320407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.320325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7" event={"ID":"9f54ee7d-9762-4394-a581-df98e2fed938","Type":"ContainerStarted","Data":"45d4daa32ab170919c483705e280d92db5f07cb2608d193f7d9e998986c422c6"}
Sep 09 03:23:57.320407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.320367    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:23:57.323021 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.322994    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:57.326491 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.326401    2580 scope.go:117] "RemoveContainer" containerID="4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c"
Sep 09 03:23:57.326900 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.326858    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c\": container with ID starting with 4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c not found: ID does not exist" containerID="4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c"
Sep 09 03:23:57.327048 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.326897    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c"} err="failed to get container status \"4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c\": rpc error: code = NotFound desc = could not find container \"4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c\": container with ID starting with 4a9bae5dee3515d5002f5db03ff90b410c841af0bf219666019429bb3ff2ae2c not found: ID does not exist"
Sep 09 03:23:57.327048 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.326923    2580 scope.go:117] "RemoveContainer" containerID="3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a"
Sep 09 03:23:57.328129 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.328092    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pww7s\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:57.330180 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.330152    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pww7s\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:57.335851 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.335834    2580 scope.go:117] "RemoveContainer" containerID="3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a"
Sep 09 03:23:57.336122 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.336090    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a\": container with ID starting with 3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a not found: ID does not exist" containerID="3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a"
Sep 09 03:23:57.336122 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.336111    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a"} err="failed to get container status \"3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a\": rpc error: code = NotFound desc = could not find container \"3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a\": container with ID starting with 3a222ff5f51d2b361d97b0dae0efff20b59386c022e40270dd6ab0946918ce6a not found: ID does not exist"
Sep 09 03:23:57.345179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.345161    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"]
Sep 09 03:23:57.351654 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:23:57.351631    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e75a48c_96cf_431e_a095_b0de7adc0039.slice/crio-84877265f802428af072e7b215e20d1438c29e73e49f6d62b4ad1f3f0665c78e WatchSource:0}: Error finding container 84877265f802428af072e7b215e20d1438c29e73e49f6d62b4ad1f3f0665c78e: Status 404 returned error can't find the container with id 84877265f802428af072e7b215e20d1438c29e73e49f6d62b4ad1f3f0665c78e
Sep 09 03:23:57.399752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.399712    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7" podStartSLOduration=1.399697899 podStartE2EDuration="1.399697899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:23:57.396603056 +0000 UTC m=+648.096367031" watchObservedRunningTime="2026-09-09 03:23:57.399697899 +0000 UTC m=+648.099461857"
Sep 09 03:23:57.403978 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.403954    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:57.407597 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.407574    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pww7s\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pww7s"
Sep 09 03:23:57.415464 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:23:57.415441    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zk5pg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zk5pg"
Sep 09 03:23:57.788954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.788921    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90014e78-9e10-46a8-9ee2-23b65e204916" path="/var/lib/kubelet/pods/90014e78-9e10-46a8-9ee2-23b65e204916/volumes"
Sep 09 03:23:57.789260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:57.789248    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c94a0938-2d81-4e30-9bea-31155074eab8" path="/var/lib/kubelet/pods/c94a0938-2d81-4e30-9bea-31155074eab8/volumes"
Sep 09 03:23:58.328944 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:58.328909    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm" event={"ID":"5e75a48c-96cf-431e-a095-b0de7adc0039","Type":"ContainerStarted","Data":"66ae9600611b49d1fb811366ddb8e8a5e0758cabfef65d2fc9d02b1897731e7b"}
Sep 09 03:23:58.329391 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:58.328968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm" event={"ID":"5e75a48c-96cf-431e-a095-b0de7adc0039","Type":"ContainerStarted","Data":"84877265f802428af072e7b215e20d1438c29e73e49f6d62b4ad1f3f0665c78e"}
Sep 09 03:23:58.402291 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:58.402246    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm" podStartSLOduration=2.402232197 podStartE2EDuration="2.402232197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:23:58.401612792 +0000 UTC m=+649.101376754" watchObservedRunningTime="2026-09-09 03:23:58.402232197 +0000 UTC m=+649.101996159"
Sep 09 03:23:59.273638 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:59.273608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-lppql"
Sep 09 03:23:59.336648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:23:59.336622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:24:00.273014 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.272973    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"]
Sep 09 03:24:00.279564 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.279542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.319697 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.319652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"]
Sep 09 03:24:00.320422 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.320402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l97hq\" (UniqueName: \"kubernetes.io/projected/2f504846-7c37-4b8c-af06-2c234fd80f48-kube-api-access-l97hq\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nx6v9\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.320484 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.320475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f504846-7c37-4b8c-af06-2c234fd80f48-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nx6v9\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.421102 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.421064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f504846-7c37-4b8c-af06-2c234fd80f48-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nx6v9\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.421523 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.421120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l97hq\" (UniqueName: \"kubernetes.io/projected/2f504846-7c37-4b8c-af06-2c234fd80f48-kube-api-access-l97hq\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nx6v9\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.421523 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.421486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f504846-7c37-4b8c-af06-2c234fd80f48-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nx6v9\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.444180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.444150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l97hq\" (UniqueName: \"kubernetes.io/projected/2f504846-7c37-4b8c-af06-2c234fd80f48-kube-api-access-l97hq\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nx6v9\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.589601 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.589535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:00.935301 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:24:00.935275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f504846_7c37_4b8c_af06_2c234fd80f48.slice/crio-718b1147364ed49eec80c5478e0efb76ab8c650562cae962e6d81577dd27bc42 WatchSource:0}: Error finding container 718b1147364ed49eec80c5478e0efb76ab8c650562cae962e6d81577dd27bc42: Status 404 returned error can't find the container with id 718b1147364ed49eec80c5478e0efb76ab8c650562cae962e6d81577dd27bc42
Sep 09 03:24:00.953199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:00.953180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"]
Sep 09 03:24:01.349603 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:01.349567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9" event={"ID":"2f504846-7c37-4b8c-af06-2c234fd80f48","Type":"ContainerStarted","Data":"0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4"}
Sep 09 03:24:01.349810 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:01.349617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9" event={"ID":"2f504846-7c37-4b8c-af06-2c234fd80f48","Type":"ContainerStarted","Data":"718b1147364ed49eec80c5478e0efb76ab8c650562cae962e6d81577dd27bc42"}
Sep 09 03:24:01.349810 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:01.349663    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:01.403231 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:01.403175    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9" podStartSLOduration=1.403161068 podStartE2EDuration="1.403161068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:24:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:24:01.400114568 +0000 UTC m=+652.099878539" watchObservedRunningTime="2026-09-09 03:24:01.403161068 +0000 UTC m=+652.102925030"
Sep 09 03:24:08.331214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:08.331186    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:24:10.344093 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:10.344066    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bxfvm"
Sep 09 03:24:12.357107 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.357079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:24:12.541256 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.541230    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"]
Sep 09 03:24:12.541467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.541444    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7" podUID="9f54ee7d-9762-4394-a581-df98e2fed938" containerName="manager" containerID="cri-o://6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5" gracePeriod=10
Sep 09 03:24:12.823707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.823664    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:24:12.920569 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.920542    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f54ee7d-9762-4394-a581-df98e2fed938-extensions-socket-volume\") pod \"9f54ee7d-9762-4394-a581-df98e2fed938\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") "
Sep 09 03:24:12.920725 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.920586    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq54f\" (UniqueName: \"kubernetes.io/projected/9f54ee7d-9762-4394-a581-df98e2fed938-kube-api-access-nq54f\") pod \"9f54ee7d-9762-4394-a581-df98e2fed938\" (UID: \"9f54ee7d-9762-4394-a581-df98e2fed938\") "
Sep 09 03:24:12.920929 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.920906    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f54ee7d-9762-4394-a581-df98e2fed938-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9f54ee7d-9762-4394-a581-df98e2fed938" (UID: "9f54ee7d-9762-4394-a581-df98e2fed938"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:24:12.922811 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:12.922762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f54ee7d-9762-4394-a581-df98e2fed938-kube-api-access-nq54f" (OuterVolumeSpecName: "kube-api-access-nq54f") pod "9f54ee7d-9762-4394-a581-df98e2fed938" (UID: "9f54ee7d-9762-4394-a581-df98e2fed938"). InnerVolumeSpecName "kube-api-access-nq54f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:24:13.021278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.021253    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f54ee7d-9762-4394-a581-df98e2fed938-extensions-socket-volume\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:24:13.021278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.021274    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq54f\" (UniqueName: \"kubernetes.io/projected/9f54ee7d-9762-4394-a581-df98e2fed938-kube-api-access-nq54f\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:24:13.415594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.415565    2580 generic.go:356] "Generic (PLEG): container finished" podID="9f54ee7d-9762-4394-a581-df98e2fed938" containerID="6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5" exitCode=0
Sep 09 03:24:13.416008 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.415608    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"
Sep 09 03:24:13.416008 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.415644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7" event={"ID":"9f54ee7d-9762-4394-a581-df98e2fed938","Type":"ContainerDied","Data":"6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5"}
Sep 09 03:24:13.416008 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.415713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7" event={"ID":"9f54ee7d-9762-4394-a581-df98e2fed938","Type":"ContainerDied","Data":"45d4daa32ab170919c483705e280d92db5f07cb2608d193f7d9e998986c422c6"}
Sep 09 03:24:13.416008 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.415745    2580 scope.go:117] "RemoveContainer" containerID="6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5"
Sep 09 03:24:13.425181 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.425165    2580 scope.go:117] "RemoveContainer" containerID="6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5"
Sep 09 03:24:13.425430 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:24:13.425411    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5\": container with ID starting with 6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5 not found: ID does not exist" containerID="6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5"
Sep 09 03:24:13.425488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.425442    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5"} err="failed to get container status \"6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5\": rpc error: code = NotFound desc = could not find container \"6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5\": container with ID starting with 6ad2cab50a753f1c8f4ab4208784360204d680ed1b4499c72f040eb8cdad95b5 not found: ID does not exist"
Sep 09 03:24:13.455068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.455043    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"]
Sep 09 03:24:13.458208 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.458189    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-59fd7"]
Sep 09 03:24:13.788244 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:13.788219    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f54ee7d-9762-4394-a581-df98e2fed938" path="/var/lib/kubelet/pods/9f54ee7d-9762-4394-a581-df98e2fed938/volumes"
Sep 09 03:24:29.416192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.416158    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"]
Sep 09 03:24:29.416639 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.416525    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f54ee7d-9762-4394-a581-df98e2fed938" containerName="manager"
Sep 09 03:24:29.416639 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.416536    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f54ee7d-9762-4394-a581-df98e2fed938" containerName="manager"
Sep 09 03:24:29.416639 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.416596    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f54ee7d-9762-4394-a581-df98e2fed938" containerName="manager"
Sep 09 03:24:29.421330 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.421315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.425071 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:24:29.425045    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.425174 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:24:29.425140    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-default-gateway-openshift-default-dockercfg-lqh2t\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lqh2t\"" type="*v1.Secret"
Sep 09 03:24:29.485371 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.485342    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"]
Sep 09 03:24:29.545318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx8rb\" (UniqueName: \"kubernetes.io/projected/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-kube-api-access-zx8rb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.545708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.545609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.646779 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.646752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.646894 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.646789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.646894 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.646821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.646964 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.646940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647003 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.646994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647047 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647103 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647150 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx8rb\" (UniqueName: \"kubernetes.io/projected/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-kube-api-access-zx8rb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647303 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647357 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647450 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.647762 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.647743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.649411 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.649388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.649587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.649569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.676793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.676739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:29.676988 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:29.676971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx8rb\" (UniqueName: \"kubernetes.io/projected/d6dcaabd-8cb8-4537-861a-fc354c8f04e0-kube-api-access-zx8rb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cgtvx\" (UID: \"d6dcaabd-8cb8-4537-861a-fc354c8f04e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:30.685916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:30.685888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lqh2t\""
Sep 09 03:24:30.691824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:30.691804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:30.991246 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:24:30.991219    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6dcaabd_8cb8_4537_861a_fc354c8f04e0.slice/crio-1add06d27393e73563fa63caf55fe7e5caf05d49bae38b73620e385c71015b1e WatchSource:0}: Error finding container 1add06d27393e73563fa63caf55fe7e5caf05d49bae38b73620e385c71015b1e: Status 404 returned error can't find the container with id 1add06d27393e73563fa63caf55fe7e5caf05d49bae38b73620e385c71015b1e
Sep 09 03:24:30.991381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:30.991360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"]
Sep 09 03:24:30.993266 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:30.993234    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:24:30.993323 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:30.993311    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:24:30.993362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:30.993342    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:24:31.515771 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:31.515734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" event={"ID":"d6dcaabd-8cb8-4537-861a-fc354c8f04e0","Type":"ContainerStarted","Data":"1d6830044fb7334ba4442ba12e6efbf4c9beee0f6e6d27b8f4abb4a2c42b0dd6"}
Sep 09 03:24:31.515937 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:31.515795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" event={"ID":"d6dcaabd-8cb8-4537-861a-fc354c8f04e0","Type":"ContainerStarted","Data":"1add06d27393e73563fa63caf55fe7e5caf05d49bae38b73620e385c71015b1e"}
Sep 09 03:24:31.558366 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:31.558324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" podStartSLOduration=2.558310884 podStartE2EDuration="2.558310884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:24:31.547960831 +0000 UTC m=+682.247724792" watchObservedRunningTime="2026-09-09 03:24:31.558310884 +0000 UTC m=+682.258074846"
Sep 09 03:24:31.692186 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:31.692162    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:32.696752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:32.696723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:33.526435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:33.526394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:33.527893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:33.527864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:24:34.162462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:34.162429    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67dv6"]
Sep 09 03:24:34.165832 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:34.165815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:34.170407 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:24:34.170373    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-7bcfd8f9cd-67dv6\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:34.170703 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:24:34.170654    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-authorino-dockercfg-7dnvg\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7dnvg\"" type="*v1.Secret"
Sep 09 03:24:34.192956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:34.192932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67dv6"]
Sep 09 03:24:34.285082 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:34.285057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gtxn\" (UniqueName: \"kubernetes.io/projected/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254-kube-api-access-4gtxn\") pod \"authorino-7bcfd8f9cd-67dv6\" (UID: \"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254\") " pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:34.385847 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:34.385822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gtxn\" (UniqueName: \"kubernetes.io/projected/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254-kube-api-access-4gtxn\") pod \"authorino-7bcfd8f9cd-67dv6\" (UID: \"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254\") " pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:34.408408 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:34.408381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gtxn\" (UniqueName: \"kubernetes.io/projected/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254-kube-api-access-4gtxn\") pod \"authorino-7bcfd8f9cd-67dv6\" (UID: \"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254\") " pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:35.474831 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:35.474798    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 09 03:24:35.475194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:35.474856    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:35.625437 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:35.625412    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7dnvg\""
Sep 09 03:24:35.756165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:35.756073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67dv6"]
Sep 09 03:24:35.758307 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:24:35.758279    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8773b9c_ac7e_4bc5_9613_e2dbd4cc5254.slice/crio-6f4a9be262274f1ce7a24d15670df67b64030bc314cf033b0983ddbcfb3f63b2 WatchSource:0}: Error finding container 6f4a9be262274f1ce7a24d15670df67b64030bc314cf033b0983ddbcfb3f63b2: Status 404 returned error can't find the container with id 6f4a9be262274f1ce7a24d15670df67b64030bc314cf033b0983ddbcfb3f63b2
Sep 09 03:24:36.547867 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:36.547826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" event={"ID":"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254","Type":"ContainerStarted","Data":"6f4a9be262274f1ce7a24d15670df67b64030bc314cf033b0983ddbcfb3f63b2"}
Sep 09 03:24:37.556122 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:37.556087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" event={"ID":"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254","Type":"ContainerStarted","Data":"6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be"}
Sep 09 03:24:37.582487 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:37.582441    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" podStartSLOduration=2.137812608 podStartE2EDuration="3.582425673s" totalImagesPullingTime="1.444613065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:24:37.577761924 +0000 UTC m=+688.277525886" watchObservedRunningTime="2026-09-09 03:24:37.582425673 +0000 UTC m=+688.282189635"
Sep 09 03:24:38.368826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:38.368794    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67dv6"]
Sep 09 03:24:39.566966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:39.566921    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" podUID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" containerName="authorino" containerID="cri-o://6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be" gracePeriod=30
Sep 09 03:24:39.809747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:39.809723    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:39.933406 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:39.933339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gtxn\" (UniqueName: \"kubernetes.io/projected/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254-kube-api-access-4gtxn\") pod \"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254\" (UID: \"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254\") "
Sep 09 03:24:39.935587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:39.935558    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254-kube-api-access-4gtxn" (OuterVolumeSpecName: "kube-api-access-4gtxn") pod "b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" (UID: "b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254"). InnerVolumeSpecName "kube-api-access-4gtxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:24:40.034462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.034437    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gtxn\" (UniqueName: \"kubernetes.io/projected/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254-kube-api-access-4gtxn\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:24:40.573693 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.573646    2580 generic.go:356] "Generic (PLEG): container finished" podID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" containerID="6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be" exitCode=0
Sep 09 03:24:40.574153 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.573700    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6"
Sep 09 03:24:40.574153 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.573713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" event={"ID":"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254","Type":"ContainerDied","Data":"6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be"}
Sep 09 03:24:40.574153 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.573764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67dv6" event={"ID":"b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254","Type":"ContainerDied","Data":"6f4a9be262274f1ce7a24d15670df67b64030bc314cf033b0983ddbcfb3f63b2"}
Sep 09 03:24:40.574153 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.573789    2580 scope.go:117] "RemoveContainer" containerID="6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be"
Sep 09 03:24:40.583257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.583242    2580 scope.go:117] "RemoveContainer" containerID="6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be"
Sep 09 03:24:40.583506 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:24:40.583490    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be\": container with ID starting with 6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be not found: ID does not exist" containerID="6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be"
Sep 09 03:24:40.583543 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.583514    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be"} err="failed to get container status \"6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be\": rpc error: code = NotFound desc = could not find container \"6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be\": container with ID starting with 6643ba590e96e48f3d7200861811c8604a41f3c09e5574cfc48aad53b96376be not found: ID does not exist"
Sep 09 03:24:40.607757 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.607735    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67dv6"]
Sep 09 03:24:40.612009 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:40.611985    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67dv6"]
Sep 09 03:24:41.789107 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:24:41.789078    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" path="/var/lib/kubelet/pods/b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254/volumes"
Sep 09 03:25:07.639908 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.639814    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-t6kn7"]
Sep 09 03:25:07.640539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.640387    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" containerName="authorino"
Sep 09 03:25:07.640539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.640406    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" containerName="authorino"
Sep 09 03:25:07.640539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.640514    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8773b9c-ac7e-4bc5-9613-e2dbd4cc5254" containerName="authorino"
Sep 09 03:25:07.643591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.643575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:07.647556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.647538    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7dnvg\""
Sep 09 03:25:07.680067 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.680049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-t6kn7"]
Sep 09 03:25:07.736940 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.736920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl9h8\" (UniqueName: \"kubernetes.io/projected/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e-kube-api-access-sl9h8\") pod \"authorino-557f876f69-t6kn7\" (UID: \"4ab9ad9c-0196-4422-b4e2-bbdfa146b84e\") " pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:07.838005 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.837978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9h8\" (UniqueName: \"kubernetes.io/projected/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e-kube-api-access-sl9h8\") pod \"authorino-557f876f69-t6kn7\" (UID: \"4ab9ad9c-0196-4422-b4e2-bbdfa146b84e\") " pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:07.853622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.853588    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-t6kn7"]
Sep 09 03:25:07.853867 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:25:07.853843    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sl9h8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-t6kn7" podUID="4ab9ad9c-0196-4422-b4e2-bbdfa146b84e"
Sep 09 03:25:07.856175 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.856156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9h8\" (UniqueName: \"kubernetes.io/projected/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e-kube-api-access-sl9h8\") pod \"authorino-557f876f69-t6kn7\" (UID: \"4ab9ad9c-0196-4422-b4e2-bbdfa146b84e\") " pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:07.914619 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.914562    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-j526d"]
Sep 09 03:25:07.917932 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.917919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:07.921216 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.921195    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 03:25:07.938733 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.938707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlhzb\" (UniqueName: \"kubernetes.io/projected/06814280-3f20-41fb-a7ee-dcd8e207f2c4-kube-api-access-mlhzb\") pod \"authorino-6d4dfc649f-j526d\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") " pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:07.938837 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.938795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06814280-3f20-41fb-a7ee-dcd8e207f2c4-tls-cert\") pod \"authorino-6d4dfc649f-j526d\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") " pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:07.940855 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:07.940833    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-j526d"]
Sep 09 03:25:08.012460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.012434    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-j526d"]
Sep 09 03:25:08.012698 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:25:08.012664    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mlhzb tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-j526d" podUID="06814280-3f20-41fb-a7ee-dcd8e207f2c4"
Sep 09 03:25:08.039713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.039656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06814280-3f20-41fb-a7ee-dcd8e207f2c4-tls-cert\") pod \"authorino-6d4dfc649f-j526d\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") " pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:08.039804 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.039748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlhzb\" (UniqueName: \"kubernetes.io/projected/06814280-3f20-41fb-a7ee-dcd8e207f2c4-kube-api-access-mlhzb\") pod \"authorino-6d4dfc649f-j526d\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") " pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:08.042062 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.042038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06814280-3f20-41fb-a7ee-dcd8e207f2c4-tls-cert\") pod \"authorino-6d4dfc649f-j526d\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") " pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:08.070019 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.070001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlhzb\" (UniqueName: \"kubernetes.io/projected/06814280-3f20-41fb-a7ee-dcd8e207f2c4-kube-api-access-mlhzb\") pod \"authorino-6d4dfc649f-j526d\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") " pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:08.725990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.725953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:08.726429 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.725954    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:08.733266 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.733248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:08.736195 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.736179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:08.744827 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.744797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl9h8\" (UniqueName: \"kubernetes.io/projected/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e-kube-api-access-sl9h8\") pod \"4ab9ad9c-0196-4422-b4e2-bbdfa146b84e\" (UID: \"4ab9ad9c-0196-4422-b4e2-bbdfa146b84e\") "
Sep 09 03:25:08.744907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.744839    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06814280-3f20-41fb-a7ee-dcd8e207f2c4-tls-cert\") pod \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") "
Sep 09 03:25:08.744907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.744875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlhzb\" (UniqueName: \"kubernetes.io/projected/06814280-3f20-41fb-a7ee-dcd8e207f2c4-kube-api-access-mlhzb\") pod \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\" (UID: \"06814280-3f20-41fb-a7ee-dcd8e207f2c4\") "
Sep 09 03:25:08.746991 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.746972    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e-kube-api-access-sl9h8" (OuterVolumeSpecName: "kube-api-access-sl9h8") pod "4ab9ad9c-0196-4422-b4e2-bbdfa146b84e" (UID: "4ab9ad9c-0196-4422-b4e2-bbdfa146b84e"). InnerVolumeSpecName "kube-api-access-sl9h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:25:08.747077 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.746988    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06814280-3f20-41fb-a7ee-dcd8e207f2c4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "06814280-3f20-41fb-a7ee-dcd8e207f2c4" (UID: "06814280-3f20-41fb-a7ee-dcd8e207f2c4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:25:08.747116 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.747080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06814280-3f20-41fb-a7ee-dcd8e207f2c4-kube-api-access-mlhzb" (OuterVolumeSpecName: "kube-api-access-mlhzb") pod "06814280-3f20-41fb-a7ee-dcd8e207f2c4" (UID: "06814280-3f20-41fb-a7ee-dcd8e207f2c4"). InnerVolumeSpecName "kube-api-access-mlhzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:25:08.845812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.845785    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl9h8\" (UniqueName: \"kubernetes.io/projected/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e-kube-api-access-sl9h8\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:25:08.845812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.845805    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06814280-3f20-41fb-a7ee-dcd8e207f2c4-tls-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:25:08.845956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:08.845817    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlhzb\" (UniqueName: \"kubernetes.io/projected/06814280-3f20-41fb-a7ee-dcd8e207f2c4-kube-api-access-mlhzb\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:25:09.731821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:09.731753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t6kn7"
Sep 09 03:25:09.731821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:09.731814    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-j526d"
Sep 09 03:25:09.790392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:09.790365    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-j526d"]
Sep 09 03:25:09.796595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:09.795587    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-j526d"]
Sep 09 03:25:09.841652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:09.841626    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-t6kn7"]
Sep 09 03:25:09.844012 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:09.843995    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-t6kn7"]
Sep 09 03:25:11.788442 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:11.788401    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06814280-3f20-41fb-a7ee-dcd8e207f2c4" path="/var/lib/kubelet/pods/06814280-3f20-41fb-a7ee-dcd8e207f2c4/volumes"
Sep 09 03:25:11.789046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:11.788724    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab9ad9c-0196-4422-b4e2-bbdfa146b84e" path="/var/lib/kubelet/pods/4ab9ad9c-0196-4422-b4e2-bbdfa146b84e/volumes"
Sep 09 03:25:50.316856 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.316821    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"]
Sep 09 03:25:50.320490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.320468    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.327467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.327447    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 03:25:50.327567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.327470    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 03:25:50.327567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.327546    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 03:25:50.328588 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.328573    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 03:25:50.427700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.427643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"]
Sep 09 03:25:50.459022 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.458995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.459133 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.459028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba48857f-4f0f-430e-b5ef-59e146ec4411-openshift-service-ca\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.459133 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.459095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-maas-api-tls\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.459133 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.459125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc62z\" (UniqueName: \"kubernetes.io/projected/ba48857f-4f0f-430e-b5ef-59e146ec4411-kube-api-access-dc62z\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.559993 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.559957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.560128 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.559997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba48857f-4f0f-430e-b5ef-59e146ec4411-openshift-service-ca\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.560128 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.560044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-maas-api-tls\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.560128 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.560074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc62z\" (UniqueName: \"kubernetes.io/projected/ba48857f-4f0f-430e-b5ef-59e146ec4411-kube-api-access-dc62z\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.560128 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:25:50.560098    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 09 03:25:50.560358 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:25:50.560158    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs podName:ba48857f-4f0f-430e-b5ef-59e146ec4411 nodeName:}" failed. No retries permitted until 2026-09-09 03:25:51.060140335 +0000 UTC m=+761.759904275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs") pod "maas-api-55d66fd5cd-9zdz7" (UID: "ba48857f-4f0f-430e-b5ef-59e146ec4411") : secret "maas-api-metrics-tls" not found
Sep 09 03:25:50.560641 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.560623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba48857f-4f0f-430e-b5ef-59e146ec4411-openshift-service-ca\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.562584 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.562566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-maas-api-tls\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:50.582826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:50.582780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc62z\" (UniqueName: \"kubernetes.io/projected/ba48857f-4f0f-430e-b5ef-59e146ec4411-kube-api-access-dc62z\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:51.065055 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:51.065019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:51.067552 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:51.067529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs\") pod \"maas-api-55d66fd5cd-9zdz7\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") " pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:51.230532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:51.230503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:51.561293 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:51.561264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"]
Sep 09 03:25:51.563884 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:25:51.563853    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba48857f_4f0f_430e_b5ef_59e146ec4411.slice/crio-4976302cfea398c74a9c430b4ec745d26fd120eec5777f7617ef3c11995401d7 WatchSource:0}: Error finding container 4976302cfea398c74a9c430b4ec745d26fd120eec5777f7617ef3c11995401d7: Status 404 returned error can't find the container with id 4976302cfea398c74a9c430b4ec745d26fd120eec5777f7617ef3c11995401d7
Sep 09 03:25:51.950626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:51.950541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7" event={"ID":"ba48857f-4f0f-430e-b5ef-59e146ec4411","Type":"ContainerStarted","Data":"4976302cfea398c74a9c430b4ec745d26fd120eec5777f7617ef3c11995401d7"}
Sep 09 03:25:53.964979 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:53.964940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7" event={"ID":"ba48857f-4f0f-430e-b5ef-59e146ec4411","Type":"ContainerStarted","Data":"25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69"}
Sep 09 03:25:53.965358 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:53.964999    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:25:54.033656 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:54.033591    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7" podStartSLOduration=2.175710299 podStartE2EDuration="4.033567937s" totalImagesPullingTime="1.857857638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:25:54.026137719 +0000 UTC m=+764.725901686" watchObservedRunningTime="2026-09-09 03:25:54.033567937 +0000 UTC m=+764.733331900"
Sep 09 03:25:59.976811 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:25:59.976784    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:26:01.006562 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:01.006531    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" containerID="1d6830044fb7334ba4442ba12e6efbf4c9beee0f6e6d27b8f4abb4a2c42b0dd6" exitCode=0
Sep 09 03:26:01.006965 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:01.006586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" event={"ID":"d6dcaabd-8cb8-4537-861a-fc354c8f04e0","Type":"ContainerDied","Data":"1d6830044fb7334ba4442ba12e6efbf4c9beee0f6e6d27b8f4abb4a2c42b0dd6"}
Sep 09 03:26:01.006965 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:01.006949    2580 scope.go:117] "RemoveContainer" containerID="1d6830044fb7334ba4442ba12e6efbf4c9beee0f6e6d27b8f4abb4a2c42b0dd6"
Sep 09 03:26:01.007866 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:01.007834    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:26:01.007970 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:01.007902    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:26:01.007970 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:01.007932    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:26:02.014469 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:02.014433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" event={"ID":"d6dcaabd-8cb8-4537-861a-fc354c8f04e0","Type":"ContainerStarted","Data":"412d074a9d4d3000627311c435eda0b1646906a37d3d0f6c70c87c4b6e025cbf"}
Sep 09 03:26:02.692868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:02.692830    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:26:02.693965 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:02.693938    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cgtvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 03:26:02.694086 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:02.694000    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" podUID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 03:26:03.527378 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:03.527342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:26:03.692621 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:03.692581    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cgtvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 03:26:03.692825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:03.692644    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" podUID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 03:26:04.692240 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:04.692204    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cgtvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 03:26:04.692732 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:04.692265    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" podUID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 03:26:05.692583 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:05.692550    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cgtvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 03:26:05.692983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:05.692605    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" podUID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 03:26:06.692729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:06.692667    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cgtvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 09 03:26:06.693101 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:06.692755    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx" podUID="d6dcaabd-8cb8-4537-861a-fc354c8f04e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 09 03:26:07.696603 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:07.696573    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:26:07.697726 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:07.697712    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cgtvx"
Sep 09 03:26:18.661385 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.661353    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"]
Sep 09 03:26:18.665468 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.665447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.674408 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.674380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 03:26:18.674537 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.674432    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9xhf2\""
Sep 09 03:26:18.674632 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.674609    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 09 03:26:18.675046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.675028    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 03:26:18.720063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.720039    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"]
Sep 09 03:26:18.792830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.792797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.792969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.792918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.792969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.792946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.793066 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.792970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4807ff-d26b-4b20-b43e-48913075dd3b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.793066 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.792987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.793066 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.793002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vtg2\" (UniqueName: \"kubernetes.io/projected/af4807ff-d26b-4b20-b43e-48913075dd3b-kube-api-access-4vtg2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.893570 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.893545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.893826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.893575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.893826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.893596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4807ff-d26b-4b20-b43e-48913075dd3b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.893826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.893617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.893826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.893642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vtg2\" (UniqueName: \"kubernetes.io/projected/af4807ff-d26b-4b20-b43e-48913075dd3b-kube-api-access-4vtg2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.893826 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.893795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.894122 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.894007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.894122 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.894113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.894233 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.894164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.896200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.896182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af4807ff-d26b-4b20-b43e-48913075dd3b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.896258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.896204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af4807ff-d26b-4b20-b43e-48913075dd3b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.906769 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.906746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vtg2\" (UniqueName: \"kubernetes.io/projected/af4807ff-d26b-4b20-b43e-48913075dd3b-kube-api-access-4vtg2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446\" (UID: \"af4807ff-d26b-4b20-b43e-48913075dd3b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:18.976150 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:18.976131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:19.301498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:19.301478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"]
Sep 09 03:26:19.303758 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:26:19.303722    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4807ff_d26b_4b20_b43e_48913075dd3b.slice/crio-08124f3011d5e8b09cf239c5b19e8706fa2bd1c4a6d0afb7a4394b7988b668e8 WatchSource:0}: Error finding container 08124f3011d5e8b09cf239c5b19e8706fa2bd1c4a6d0afb7a4394b7988b668e8: Status 404 returned error can't find the container with id 08124f3011d5e8b09cf239c5b19e8706fa2bd1c4a6d0afb7a4394b7988b668e8
Sep 09 03:26:20.125340 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:20.125306    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446" event={"ID":"af4807ff-d26b-4b20-b43e-48913075dd3b","Type":"ContainerStarted","Data":"08124f3011d5e8b09cf239c5b19e8706fa2bd1c4a6d0afb7a4394b7988b668e8"}
Sep 09 03:26:21.131922 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:21.131890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446" event={"ID":"af4807ff-d26b-4b20-b43e-48913075dd3b","Type":"ContainerStarted","Data":"ea15d5060a92fa04726906f9d29245466680bfdb453de01e248c48c23b81a4e0"}
Sep 09 03:26:21.196392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:21.196340    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446" podStartSLOduration=2.080421516 podStartE2EDuration="3.196325773s" totalImagesPullingTime="1.115904257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:26:21.193039155 +0000 UTC m=+791.892803118" watchObservedRunningTime="2026-09-09 03:26:21.196325773 +0000 UTC m=+791.896089734"
Sep 09 03:26:28.976612 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:28.976579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:28.977009 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:28.976636    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:28.989442 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:28.989424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:29.192114 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:29.192084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446"
Sep 09 03:26:59.622157 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.622115    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"]
Sep 09 03:26:59.632530 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.632504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.637322 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.637295    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 09 03:26:59.711957 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.711929    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"]
Sep 09 03:26:59.734804 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.734774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdnsm\" (UniqueName: \"kubernetes.io/projected/86233792-c185-4d21-bf09-6f660873f901-kube-api-access-bdnsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.734942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.734835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86233792-c185-4d21-bf09-6f660873f901-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.734942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.734879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.735058 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.734946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.735058 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.734983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.735158 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.735062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835628 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835628 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdnsm\" (UniqueName: \"kubernetes.io/projected/86233792-c185-4d21-bf09-6f660873f901-kube-api-access-bdnsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835628 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86233792-c185-4d21-bf09-6f660873f901-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.835970 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.835945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.836087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.836007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.836087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.836075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.837747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.837728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86233792-c185-4d21-bf09-6f660873f901-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.838095 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.838077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86233792-c185-4d21-bf09-6f660873f901-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.855838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.855815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdnsm\" (UniqueName: \"kubernetes.io/projected/86233792-c185-4d21-bf09-6f660873f901-kube-api-access-bdnsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7lkbm\" (UID: \"86233792-c185-4d21-bf09-6f660873f901\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:26:59.944261 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:26:59.944189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:27:00.244663 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:00.244641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"]
Sep 09 03:27:00.245954 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:27:00.245929    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86233792_c185_4d21_bf09_6f660873f901.slice/crio-3c3373c7166b162372294eb702e38c6f7fe8669ba1289ea67505d3c89b07ee7c WatchSource:0}: Error finding container 3c3373c7166b162372294eb702e38c6f7fe8669ba1289ea67505d3c89b07ee7c: Status 404 returned error can't find the container with id 3c3373c7166b162372294eb702e38c6f7fe8669ba1289ea67505d3c89b07ee7c
Sep 09 03:27:00.349125 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:00.349088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm" event={"ID":"86233792-c185-4d21-bf09-6f660873f901","Type":"ContainerStarted","Data":"3c3373c7166b162372294eb702e38c6f7fe8669ba1289ea67505d3c89b07ee7c"}
Sep 09 03:27:01.355909 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:01.355871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm" event={"ID":"86233792-c185-4d21-bf09-6f660873f901","Type":"ContainerStarted","Data":"140113cccae3b1f2222fac139ffb3638a99e17a0ce4d2549e6f47c6edb6e8010"}
Sep 09 03:27:01.387403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:01.387351    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm" podStartSLOduration=2.213107104 podStartE2EDuration="2.387336201s" totalImagesPullingTime="174.229097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:27:01.384016349 +0000 UTC m=+832.083780311" watchObservedRunningTime="2026-09-09 03:27:01.387336201 +0000 UTC m=+832.087100166"
Sep 09 03:27:09.944887 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:09.944855    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:27:09.945264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:09.945162    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:27:09.959064 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:09.959042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:27:10.416509 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:10.416481    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7lkbm"
Sep 09 03:27:24.774787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.774756    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"]
Sep 09 03:27:24.778355 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.778340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:24.785585 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.785555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 09 03:27:24.838217 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.838190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"]
Sep 09 03:27:24.935255 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.935225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:24.935403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.935267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:24.935403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.935287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:24.935528 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.935407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:24.935528 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.935455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55bff30a-8a10-4730-857b-851339e3266f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:24.935633 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:24.935536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzzwx\" (UniqueName: \"kubernetes.io/projected/55bff30a-8a10-4730-857b-851339e3266f-kube-api-access-wzzwx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036559 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036559 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55bff30a-8a10-4730-857b-851339e3266f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036559 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzzwx\" (UniqueName: \"kubernetes.io/projected/55bff30a-8a10-4730-857b-851339e3266f-kube-api-access-wzzwx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036825 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.036921 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.036881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.038861 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.038829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55bff30a-8a10-4730-857b-851339e3266f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.039060 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.039042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55bff30a-8a10-4730-857b-851339e3266f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.060773 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.060751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzzwx\" (UniqueName: \"kubernetes.io/projected/55bff30a-8a10-4730-857b-851339e3266f-kube-api-access-wzzwx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4\" (UID: \"55bff30a-8a10-4730-857b-851339e3266f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.087800 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.087780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:25.376501 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.376475    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"]
Sep 09 03:27:25.378770 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:27:25.378741    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55bff30a_8a10_4730_857b_851339e3266f.slice/crio-9f86518e4990b3777ced04a873f6094da85c354d36090dfd6b7cb2bee9e43e18 WatchSource:0}: Error finding container 9f86518e4990b3777ced04a873f6094da85c354d36090dfd6b7cb2bee9e43e18: Status 404 returned error can't find the container with id 9f86518e4990b3777ced04a873f6094da85c354d36090dfd6b7cb2bee9e43e18
Sep 09 03:27:25.496551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:25.496521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4" event={"ID":"55bff30a-8a10-4730-857b-851339e3266f","Type":"ContainerStarted","Data":"9f86518e4990b3777ced04a873f6094da85c354d36090dfd6b7cb2bee9e43e18"}
Sep 09 03:27:26.504838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:26.504807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4" event={"ID":"55bff30a-8a10-4730-857b-851339e3266f","Type":"ContainerStarted","Data":"4c006efd8dc3ce76de8f4343e5c9c126bdfe5aed9d408d1ad7eb84845a7098eb"}
Sep 09 03:27:26.541694 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:26.541635    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4" podStartSLOduration=2.347188213 podStartE2EDuration="2.541621066s" totalImagesPullingTime="194.432853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:27:26.540007147 +0000 UTC m=+857.239771112" watchObservedRunningTime="2026-09-09 03:27:26.541621066 +0000 UTC m=+857.241385028"
Sep 09 03:27:35.088876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:35.088833    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:35.088876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:35.088887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:35.101030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:35.101007    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:27:35.571433 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:27:35.571408    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4"
Sep 09 03:29:16.321934 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.321899    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"]
Sep 09 03:29:16.325499 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.325480    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.335964 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.335945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-40a1c9-openshift-default-dockercfg-llfg6\""
Sep 09 03:29:16.366268 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.366239    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"]
Sep 09 03:29:16.508098 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj75d\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-kube-api-access-fj75d\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-token\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-certs\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-socket\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-credential-socket\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-envoy\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508475 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-podinfo\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508475 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istiod-ca-cert\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.508475 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.508389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-data\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609568 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-podinfo\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609568 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istiod-ca-cert\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609568 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-data\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj75d\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-kube-api-access-fj75d\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-token\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-certs\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-socket\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-credential-socket\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.609947 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.609749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-envoy\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.610275 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.610099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-data\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.610275 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.610134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-socket\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.610275 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.610193    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-credential-socket\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.610408 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.610388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-certs\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.610592 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.610570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istiod-ca-cert\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.612129 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.612106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-envoy\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.612233 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.612199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-podinfo\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.625694 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.625644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj75d\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-kube-api-access-fj75d\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.629085 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.629058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-token\") pod \"e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") " pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.637753 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.637729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:16.941203 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.941117    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"]
Sep 09 03:29:16.950080 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:29:16.950052    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f8ad20_0d73_47ef_ad8e_41da0c987e47.slice/crio-6cd3b9af64d0a7f0163cc776323e0dcbbb02822baa69884fa0cf0b6c5cdafa89 WatchSource:0}: Error finding container 6cd3b9af64d0a7f0163cc776323e0dcbbb02822baa69884fa0cf0b6c5cdafa89: Status 404 returned error can't find the container with id 6cd3b9af64d0a7f0163cc776323e0dcbbb02822baa69884fa0cf0b6c5cdafa89
Sep 09 03:29:16.953131 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.953091    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:16.953257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.953166    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:16.953257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:16.953194    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:17.131496 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:17.131457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" event={"ID":"57f8ad20-0d73-47ef-ad8e-41da0c987e47","Type":"ContainerStarted","Data":"4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"}
Sep 09 03:29:17.131710 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:17.131522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" event={"ID":"57f8ad20-0d73-47ef-ad8e-41da0c987e47","Type":"ContainerStarted","Data":"6cd3b9af64d0a7f0163cc776323e0dcbbb02822baa69884fa0cf0b6c5cdafa89"}
Sep 09 03:29:17.174994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:17.174937    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podStartSLOduration=1.174921397 podStartE2EDuration="1.174921397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:17.173334306 +0000 UTC m=+967.873098265" watchObservedRunningTime="2026-09-09 03:29:17.174921397 +0000 UTC m=+967.874685360"
Sep 09 03:29:17.638699 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:17.638657    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:18.643381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:18.643351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:19.142811 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:19.142780    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:19.143684 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:19.143649    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:20.758088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.758049    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"]
Sep 09 03:29:20.762858 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.762837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.767105 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.767059    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-phvzt\""
Sep 09 03:29:20.767850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.767828    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-40a1c9-serving-cert\""
Sep 09 03:29:20.794787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.794755    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"]
Sep 09 03:29:20.848926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.848878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.849120 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.849095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a9a1044-a84b-47b3-b187-0f7831119ed6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.849199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.849143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-metrics-certs\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.849270 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.849229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82d62\" (UniqueName: \"kubernetes.io/projected/0a9a1044-a84b-47b3-b187-0f7831119ed6-kube-api-access-82d62\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.950688 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.950647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a9a1044-a84b-47b3-b187-0f7831119ed6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.950868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.950697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-metrics-certs\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.950868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.950745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82d62\" (UniqueName: \"kubernetes.io/projected/0a9a1044-a84b-47b3-b187-0f7831119ed6-kube-api-access-82d62\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.950868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.950793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.951050 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:29:20.950930    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-serving-cert: secret "maas-api-e2e-inf-a-40a1c9-serving-cert" not found
Sep 09 03:29:20.951050 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:29:20.951002    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls podName:0a9a1044-a84b-47b3-b187-0f7831119ed6 nodeName:}" failed. No retries permitted until 2026-09-09 03:29:21.450979974 +0000 UTC m=+972.150743928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls") pod "maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" (UID: "0a9a1044-a84b-47b3-b187-0f7831119ed6") : secret "maas-api-e2e-inf-a-40a1c9-serving-cert" not found
Sep 09 03:29:20.951471 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.951446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a9a1044-a84b-47b3-b187-0f7831119ed6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.953542 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.953511    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-metrics-certs\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:20.971090 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:20.971070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82d62\" (UniqueName: \"kubernetes.io/projected/0a9a1044-a84b-47b3-b187-0f7831119ed6-kube-api-access-82d62\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:21.455627 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:21.455598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:21.458541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:21.458510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls\") pod \"maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:21.679727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:21.679664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:21.966104 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:21.966037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"]
Sep 09 03:29:21.968780 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:29:21.968738    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9a1044_a84b_47b3_b187_0f7831119ed6.slice/crio-f2fb71872e6d4412ff6179268dc36c7c17a4cf5c667c04e7c60e183bbdd8a687 WatchSource:0}: Error finding container f2fb71872e6d4412ff6179268dc36c7c17a4cf5c667c04e7c60e183bbdd8a687: Status 404 returned error can't find the container with id f2fb71872e6d4412ff6179268dc36c7c17a4cf5c667c04e7c60e183bbdd8a687
Sep 09 03:29:21.970309 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:21.970289    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:29:22.160281 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:22.160201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" event={"ID":"0a9a1044-a84b-47b3-b187-0f7831119ed6","Type":"ContainerStarted","Data":"f2fb71872e6d4412ff6179268dc36c7c17a4cf5c667c04e7c60e183bbdd8a687"}
Sep 09 03:29:23.167750 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:23.167721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" event={"ID":"0a9a1044-a84b-47b3-b187-0f7831119ed6","Type":"ContainerStarted","Data":"4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f"}
Sep 09 03:29:23.168119 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:23.167823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:23.219252 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:23.219208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" podStartSLOduration=2.730760036 podStartE2EDuration="3.219193257s" totalImagesPullingTime="488.433221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:23.21715976 +0000 UTC m=+973.916923722" watchObservedRunningTime="2026-09-09 03:29:23.219193257 +0000 UTC m=+973.918957218"
Sep 09 03:29:25.828857 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.828820    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"]
Sep 09 03:29:25.832845 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.832827    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.836382 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.836364    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-a2fc47-openshift-default-dockercfg-nmknx\""
Sep 09 03:29:25.857522 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.857496    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"]
Sep 09 03:29:25.894416 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-data\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-socket\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-envoy\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istiod-ca-cert\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-credential-socket\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt2wq\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-kube-api-access-zt2wq\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-token\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-podinfo\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.894738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.894720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-certs\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.995994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-credential-socket\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt2wq\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-kube-api-access-zt2wq\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-token\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-podinfo\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-certs\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-data\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-socket\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-envoy\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istiod-ca-cert\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996657 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-credential-socket\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996657 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-certs\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996657 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-data\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996799 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-socket\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.996895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.996878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istiod-ca-cert\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.998643 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.998618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-envoy\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:25.998801 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:25.998780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-podinfo\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:26.010514 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.010489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-token\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:26.011362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.011339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt2wq\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-kube-api-access-zt2wq\") pod \"e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") " pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:26.142973 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.142899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:26.449181 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.449147    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"]
Sep 09 03:29:26.449990 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:29:26.449968    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24519fc_54ef_42c2_962f_a1cf6a0b9367.slice/crio-894769b3e74194757cb98e017de9f91da9e64142dc4f81d3eee955c2ca5f8830 WatchSource:0}: Error finding container 894769b3e74194757cb98e017de9f91da9e64142dc4f81d3eee955c2ca5f8830: Status 404 returned error can't find the container with id 894769b3e74194757cb98e017de9f91da9e64142dc4f81d3eee955c2ca5f8830
Sep 09 03:29:26.452447 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.452412    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:26.452519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.452501    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:26.452568 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:26.452551    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:27.193571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:27.193536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" event={"ID":"a24519fc-54ef-42c2-962f-a1cf6a0b9367","Type":"ContainerStarted","Data":"e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"}
Sep 09 03:29:27.194059 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:27.193589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" event={"ID":"a24519fc-54ef-42c2-962f-a1cf6a0b9367","Type":"ContainerStarted","Data":"894769b3e74194757cb98e017de9f91da9e64142dc4f81d3eee955c2ca5f8830"}
Sep 09 03:29:27.231360 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:27.231299    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podStartSLOduration=2.231279814 podStartE2EDuration="2.231279814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:27.230932776 +0000 UTC m=+977.930696739" watchObservedRunningTime="2026-09-09 03:29:27.231279814 +0000 UTC m=+977.931043777"
Sep 09 03:29:28.145301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:28.145268    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:28.155154 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:28.155128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:28.199533 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:28.199500    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:28.200438 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:28.200418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:29.179701 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:29.179651    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:29:35.184821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:35.184778    2580 patch_prober.go:28] interesting pod/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 03:29:35.185206 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:35.184832    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 03:29:36.252245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:36.252207    2580 generic.go:356] "Generic (PLEG): container finished" podID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerID="4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723" exitCode=0
Sep 09 03:29:36.252690 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:36.252280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" event={"ID":"57f8ad20-0d73-47ef-ad8e-41da0c987e47","Type":"ContainerDied","Data":"4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"}
Sep 09 03:29:36.252690 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:36.252662    2580 scope.go:117] "RemoveContainer" containerID="4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"
Sep 09 03:29:36.253780 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:36.253751    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:36.253852 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:36.253836    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:36.253893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:36.253878    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:37.261698 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:37.261651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" event={"ID":"57f8ad20-0d73-47ef-ad8e-41da0c987e47","Type":"ContainerStarted","Data":"34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9"}
Sep 09 03:29:37.638264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:37.638168    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:37.638482 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:37.638456    2580 patch_prober.go:28] interesting pod/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 03:29:37.638586 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:37.638514    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 03:29:38.638700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:38.638639    2580 patch_prober.go:28] interesting pod/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 03:29:38.639287 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:38.639253    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 03:29:39.638558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:39.638527    2580 patch_prober.go:28] interesting pod/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 03:29:39.638875 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:39.638846    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 03:29:40.639030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:40.638971    2580 patch_prober.go:28] interesting pod/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 03:29:40.639522 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:40.639070    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 03:29:41.638684 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:41.638631    2580 patch_prober.go:28] interesting pod/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 03:29:41.639007 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:41.638955    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 03:29:42.653201 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:42.653169    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:42.654317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:42.654297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:42.655517 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:42.655479    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:29:50.182898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.182862    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"]
Sep 09 03:29:50.185456 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.185441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.188626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.188607    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4bd6b8-openshift-default-dockercfg-rmr6z\""
Sep 09 03:29:50.202267 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.202246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"]
Sep 09 03:29:50.351624 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.351593    2580 generic.go:356] "Generic (PLEG): container finished" podID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerID="e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91" exitCode=0
Sep 09 03:29:50.351805 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.351697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" event={"ID":"a24519fc-54ef-42c2-962f-a1cf6a0b9367","Type":"ContainerDied","Data":"e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"}
Sep 09 03:29:50.352117 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.352100    2580 scope.go:117] "RemoveContainer" containerID="e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"
Sep 09 03:29:50.353202 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.353175    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:50.353266 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.353242    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:50.353301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.353275    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:50.367922 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.367898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krj4t\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-kube-api-access-krj4t\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368038 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.367928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-socket\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368038 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.367947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-token\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368038 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.368024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-credential-socket\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.368050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-envoy\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.368085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-data\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.368178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-certs\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.368194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istiod-ca-cert\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.368442 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.368270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-podinfo\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-data\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-certs\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istiod-ca-cert\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469725    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-data\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-podinfo\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469950 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krj4t\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-kube-api-access-krj4t\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469950 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-socket\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.469950 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-token\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.470107 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.469980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-credential-socket\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.470107 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.470014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-envoy\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.470348 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.470325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istiod-ca-cert\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.470602 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.470579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-socket\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.471160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.471134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-certs\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.471542 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.471508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-credential-socket\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.473763 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.473746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-podinfo\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.474413 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.474394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-envoy\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.480467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.480443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-token\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.483387 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.483281    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krj4t\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-kube-api-access-krj4t\") pod \"e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") " pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.495352 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.495326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:50.787711 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.787609    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"]
Sep 09 03:29:50.790522 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:29:50.790491    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1930e3c_390d_44bd_87d7_03ead6aa48e7.slice/crio-523c5b216624e2888f95c9bfee46d932f45ec86b68da29da6262d33eb64225ca WatchSource:0}: Error finding container 523c5b216624e2888f95c9bfee46d932f45ec86b68da29da6262d33eb64225ca: Status 404 returned error can't find the container with id 523c5b216624e2888f95c9bfee46d932f45ec86b68da29da6262d33eb64225ca
Sep 09 03:29:50.793007 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.792977    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:50.793079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.793040    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:50.793079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:50.793068    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:29:51.363754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.363715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" event={"ID":"a24519fc-54ef-42c2-962f-a1cf6a0b9367","Type":"ContainerStarted","Data":"7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd"}
Sep 09 03:29:51.365562 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.365539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" event={"ID":"e1930e3c-390d-44bd-87d7-03ead6aa48e7","Type":"ContainerStarted","Data":"0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"}
Sep 09 03:29:51.365716 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.365578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" event={"ID":"e1930e3c-390d-44bd-87d7-03ead6aa48e7","Type":"ContainerStarted","Data":"523c5b216624e2888f95c9bfee46d932f45ec86b68da29da6262d33eb64225ca"}
Sep 09 03:29:51.407143 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.407095    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podStartSLOduration=1.4070808160000001 podStartE2EDuration="1.407080816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:29:51.405857096 +0000 UTC m=+1002.105621062" watchObservedRunningTime="2026-09-09 03:29:51.407080816 +0000 UTC m=+1002.106844778"
Sep 09 03:29:51.496844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.496810    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:51.498503 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.498481    2580 patch_prober.go:28] interesting pod/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:29:51.498578 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:51.498531    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 09 03:29:52.143899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:52.143867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:52.144214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:52.144194    2580 patch_prober.go:28] interesting pod/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 03:29:52.144351 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:52.144329    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 03:29:52.499828 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:52.499805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:53.143953 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:53.143916    2580 patch_prober.go:28] interesting pod/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 03:29:53.144245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:53.144216    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 03:29:53.379364 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:53.379338    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:53.380389 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:53.380371    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:29:54.143800 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:54.143752    2580 patch_prober.go:28] interesting pod/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 03:29:54.144824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:54.144381    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 03:29:55.143913 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:55.143877    2580 patch_prober.go:28] interesting pod/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 03:29:55.144492 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:55.144455    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 03:29:56.144269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:56.144225    2580 patch_prober.go:28] interesting pod/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 09 03:29:56.144776 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:56.144293    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 09 03:29:57.148035 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:57.148002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:57.148436 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:57.148293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:29:57.148966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:29:57.148951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:30:00.148189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.148157    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"]
Sep 09 03:30:00.150975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.150958    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"]
Sep 09 03:30:00.151083 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.151065    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"
Sep 09 03:30:00.153433 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.153411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:30:00.153651 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.153635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"
Sep 09 03:30:00.162613 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.162591    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"
Sep 09 03:30:00.169146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.169126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"]
Sep 09 03:30:00.171518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.171496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"
Sep 09 03:30:00.174005 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.173984    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"]
Sep 09 03:30:00.454715 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.454684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"]
Sep 09 03:30:00.455993 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:30:00.455967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e9367f0_913e_458b_ac17_d075d5c3ac48.slice/crio-a6330c92dd806c5a4723d759b6c90380e71e2a1cce79d114f54bdbc68f33137f WatchSource:0}: Error finding container a6330c92dd806c5a4723d759b6c90380e71e2a1cce79d114f54bdbc68f33137f: Status 404 returned error can't find the container with id a6330c92dd806c5a4723d759b6c90380e71e2a1cce79d114f54bdbc68f33137f
Sep 09 03:30:00.472941 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:00.472836    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"]
Sep 09 03:30:00.473595 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:30:00.473542    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60957e1c_8024_400f_a030_2fdda570b0fe.slice/crio-da937909a0f3bf929fa9fb14cf804be1e6888c41c60853342aca85a16454f841 WatchSource:0}: Error finding container da937909a0f3bf929fa9fb14cf804be1e6888c41c60853342aca85a16454f841: Status 404 returned error can't find the container with id da937909a0f3bf929fa9fb14cf804be1e6888c41c60853342aca85a16454f841
Sep 09 03:30:01.433646 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:01.433604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56" event={"ID":"60957e1c-8024-400f-a030-2fdda570b0fe","Type":"ContainerStarted","Data":"da937909a0f3bf929fa9fb14cf804be1e6888c41c60853342aca85a16454f841"}
Sep 09 03:30:01.436017 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:01.435979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf" event={"ID":"7e9367f0-913e-458b-ac17-d075d5c3ac48","Type":"ContainerStarted","Data":"a6330c92dd806c5a4723d759b6c90380e71e2a1cce79d114f54bdbc68f33137f"}
Sep 09 03:30:03.452211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:03.452124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf" event={"ID":"7e9367f0-913e-458b-ac17-d075d5c3ac48","Type":"ContainerStarted","Data":"e40a4af1960a9fed40675a64f3419d10955859f18d66194f7faef3f88aac85a3"}
Sep 09 03:30:03.453836 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:03.453813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56" event={"ID":"60957e1c-8024-400f-a030-2fdda570b0fe","Type":"ContainerStarted","Data":"65788b4aa487718bda5f2077e4b5d0138ae039862c24ac398284561fbd0fcefd"}
Sep 09 03:30:03.518726 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:03.518683    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56" podStartSLOduration=1.144719395 podStartE2EDuration="3.518656042s" totalImagesPullingTime="2.373936647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:03.51596928 +0000 UTC m=+1014.215733242" watchObservedRunningTime="2026-09-09 03:30:03.518656042 +0000 UTC m=+1014.218420003"
Sep 09 03:30:03.519412 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:03.519387    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf" podStartSLOduration=1.131575825 podStartE2EDuration="3.51938015s" totalImagesPullingTime="2.387804325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:30:03.487295649 +0000 UTC m=+1014.187059622" watchObservedRunningTime="2026-09-09 03:30:03.51938015 +0000 UTC m=+1014.219144135"
Sep 09 03:30:04.461793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:04.461754    2580 generic.go:356] "Generic (PLEG): container finished" podID="7e9367f0-913e-458b-ac17-d075d5c3ac48" containerID="e40a4af1960a9fed40675a64f3419d10955859f18d66194f7faef3f88aac85a3" exitCode=0
Sep 09 03:30:04.462170 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:04.461836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf" event={"ID":"7e9367f0-913e-458b-ac17-d075d5c3ac48","Type":"ContainerDied","Data":"e40a4af1960a9fed40675a64f3419d10955859f18d66194f7faef3f88aac85a3"}
Sep 09 03:30:04.463620 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:04.463596    2580 generic.go:356] "Generic (PLEG): container finished" podID="60957e1c-8024-400f-a030-2fdda570b0fe" containerID="65788b4aa487718bda5f2077e4b5d0138ae039862c24ac398284561fbd0fcefd" exitCode=0
Sep 09 03:30:04.463759 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:04.463632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56" event={"ID":"60957e1c-8024-400f-a030-2fdda570b0fe","Type":"ContainerDied","Data":"65788b4aa487718bda5f2077e4b5d0138ae039862c24ac398284561fbd0fcefd"}
Sep 09 03:30:05.607956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:05.607926    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"
Sep 09 03:30:05.653358 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:05.653335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"
Sep 09 03:30:06.476563 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:06.476532    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56"
Sep 09 03:30:06.476789 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:06.476567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815410-ldr56" event={"ID":"60957e1c-8024-400f-a030-2fdda570b0fe","Type":"ContainerDied","Data":"da937909a0f3bf929fa9fb14cf804be1e6888c41c60853342aca85a16454f841"}
Sep 09 03:30:06.476789 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:06.476615    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da937909a0f3bf929fa9fb14cf804be1e6888c41c60853342aca85a16454f841"
Sep 09 03:30:06.478005 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:06.477986    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"
Sep 09 03:30:06.478121 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:06.478007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf" event={"ID":"7e9367f0-913e-458b-ac17-d075d5c3ac48","Type":"ContainerDied","Data":"a6330c92dd806c5a4723d759b6c90380e71e2a1cce79d114f54bdbc68f33137f"}
Sep 09 03:30:06.478121 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:06.478038    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6330c92dd806c5a4723d759b6c90380e71e2a1cce79d114f54bdbc68f33137f"
Sep 09 03:30:17.547770 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:17.547736    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerID="0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715" exitCode=0
Sep 09 03:30:17.548167 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:17.547813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" event={"ID":"e1930e3c-390d-44bd-87d7-03ead6aa48e7","Type":"ContainerDied","Data":"0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"}
Sep 09 03:30:17.548240 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:17.548225    2580 scope.go:117] "RemoveContainer" containerID="0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"
Sep 09 03:30:17.549339 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:17.549306    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:30:17.549485 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:17.549398    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:30:17.549485 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:17.549445    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:30:18.558107 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:18.558075    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" event={"ID":"e1930e3c-390d-44bd-87d7-03ead6aa48e7","Type":"ContainerStarted","Data":"69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a"}
Sep 09 03:30:19.496533 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:19.496493    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:30:19.497494 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:19.497467    2580 patch_prober.go:28] interesting pod/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:30:19.497615 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:19.497517    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 09 03:30:20.496889 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:20.496856    2580 patch_prober.go:28] interesting pod/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:30:20.497360 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:20.496921    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 09 03:30:21.496649 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:21.496616    2580 patch_prober.go:28] interesting pod/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:30:21.496852 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:21.496695    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 09 03:30:22.496641 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:22.496606    2580 patch_prober.go:28] interesting pod/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:30:22.497044 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:22.496660    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 09 03:30:23.380155 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:23.380121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:30:23.496111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:23.496078    2580 patch_prober.go:28] interesting pod/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 09 03:30:23.496272 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:23.496128    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 09 03:30:24.501407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:24.501374    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:30:24.502741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:30:24.502706    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:31:29.891331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891257    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"]
Sep 09 03:31:29.891812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891703    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60957e1c-8024-400f-a030-2fdda570b0fe" containerName="cleanup"
Sep 09 03:31:29.891812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891715    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="60957e1c-8024-400f-a030-2fdda570b0fe" containerName="cleanup"
Sep 09 03:31:29.891812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891731    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9367f0-913e-458b-ac17-d075d5c3ac48" containerName="cleanup"
Sep 09 03:31:29.891812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891736    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9367f0-913e-458b-ac17-d075d5c3ac48" containerName="cleanup"
Sep 09 03:31:29.891812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891797    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e9367f0-913e-458b-ac17-d075d5c3ac48" containerName="cleanup"
Sep 09 03:31:29.891812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.891807    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="60957e1c-8024-400f-a030-2fdda570b0fe" containerName="cleanup"
Sep 09 03:31:29.894994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.894979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:29.898842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.898826    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ad343832-dockercfg-w4lrv\""
Sep 09 03:31:29.899313 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.899298    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ad343832\""
Sep 09 03:31:29.915731 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.915658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"]
Sep 09 03:31:29.999058 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.999026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxmtx\" (UniqueName: \"kubernetes.io/projected/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-kube-api-access-nxmtx\") pod \"payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:29.999229 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:29.999064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:30.100049 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:30.100019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxmtx\" (UniqueName: \"kubernetes.io/projected/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-kube-api-access-nxmtx\") pod \"payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:30.100184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:30.100075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:30.101008 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:30.100990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:30.110215 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:30.110193    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxmtx\" (UniqueName: \"kubernetes.io/projected/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-kube-api-access-nxmtx\") pod \"payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:30.212432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:30.212374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:30.490778 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:30.490754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"]
Sep 09 03:31:30.492749 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:31:30.492719    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7a19d8_19ad_4870_b454_a9a2dd9e6136.slice/crio-ceb5f672269576fdc3a3f09f2cfc7af2d12a623f45c037e1fee94140556b1283 WatchSource:0}: Error finding container ceb5f672269576fdc3a3f09f2cfc7af2d12a623f45c037e1fee94140556b1283: Status 404 returned error can't find the container with id ceb5f672269576fdc3a3f09f2cfc7af2d12a623f45c037e1fee94140556b1283
Sep 09 03:31:31.001549 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:31.001515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw" event={"ID":"0a7a19d8-19ad-4870-b454-a9a2dd9e6136","Type":"ContainerStarted","Data":"ceb5f672269576fdc3a3f09f2cfc7af2d12a623f45c037e1fee94140556b1283"}
Sep 09 03:31:33.018529 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:33.018491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw" event={"ID":"0a7a19d8-19ad-4870-b454-a9a2dd9e6136","Type":"ContainerStarted","Data":"f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6"}
Sep 09 03:31:33.018996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:33.018580    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:44.028205 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:44.028173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:44.072297 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:44.072251    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw" podStartSLOduration=13.324156487 podStartE2EDuration="15.072235549s" totalImagesPullingTime="1.748079062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:31:33.037489281 +0000 UTC m=+1103.737253241" watchObservedRunningTime="2026-09-09 03:31:44.072235549 +0000 UTC m=+1114.771999511"
Sep 09 03:31:52.393623 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.393593    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"]
Sep 09 03:31:52.397332 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.397313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.400301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.400279    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-e893b71c-kserve-self-signed-certs\""
Sep 09 03:31:52.407481 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.407459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"]
Sep 09 03:31:52.511439 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.511408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1170ada0-488b-4f01-8add-33426181954e-tls-certs\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.511589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.511515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-model-cache\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.511589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.511540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-tmp-dir\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.511589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.511567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-dshm\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.511759 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.511588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlldp\" (UniqueName: \"kubernetes.io/projected/1170ada0-488b-4f01-8add-33426181954e-kube-api-access-nlldp\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.511759 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.511610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-home\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612512 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-model-cache\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-tmp-dir\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-dshm\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlldp\" (UniqueName: \"kubernetes.io/projected/1170ada0-488b-4f01-8add-33426181954e-kube-api-access-nlldp\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-home\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1170ada0-488b-4f01-8add-33426181954e-tls-certs\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.612970 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-tmp-dir\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.613036 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.612964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-model-cache\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.613093 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.613076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-home\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.614931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.614909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-dshm\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.615186 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.615169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1170ada0-488b-4f01-8add-33426181954e-tls-certs\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.621914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.621891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlldp\" (UniqueName: \"kubernetes.io/projected/1170ada0-488b-4f01-8add-33426181954e-kube-api-access-nlldp\") pod \"e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") " pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.707621 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.707538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:31:52.983817 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:52.983789    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"]
Sep 09 03:31:52.984747 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:31:52.984723    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1170ada0_488b_4f01_8add_33426181954e.slice/crio-db0777f5b951f9e1ff5f24922fea340fcdec242e1ec9130db7bfbd944f890ee3 WatchSource:0}: Error finding container db0777f5b951f9e1ff5f24922fea340fcdec242e1ec9130db7bfbd944f890ee3: Status 404 returned error can't find the container with id db0777f5b951f9e1ff5f24922fea340fcdec242e1ec9130db7bfbd944f890ee3
Sep 09 03:31:53.146534 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:53.146494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p" event={"ID":"1170ada0-488b-4f01-8add-33426181954e","Type":"ContainerStarted","Data":"630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5"}
Sep 09 03:31:53.146721 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:53.146542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p" event={"ID":"1170ada0-488b-4f01-8add-33426181954e","Type":"ContainerStarted","Data":"db0777f5b951f9e1ff5f24922fea340fcdec242e1ec9130db7bfbd944f890ee3"}
Sep 09 03:31:59.335247 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.335169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p" podStartSLOduration=7.335144203 podStartE2EDuration="7.335144203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:31:53.188960471 +0000 UTC m=+1123.888724444" watchObservedRunningTime="2026-09-09 03:31:59.335144203 +0000 UTC m=+1130.034908166"
Sep 09 03:31:59.337459 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.337422    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"]
Sep 09 03:31:59.338438 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.337917    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw" podUID="0a7a19d8-19ad-4870-b454-a9a2dd9e6136" containerName="payload-pre-processing" containerID="cri-o://f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6" gracePeriod=30
Sep 09 03:31:59.608695 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.608646    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:31:59.785150 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.785119    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxmtx\" (UniqueName: \"kubernetes.io/projected/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-kube-api-access-nxmtx\") pod \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") "
Sep 09 03:31:59.785339 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.785190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-plugins-config-volume\") pod \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\" (UID: \"0a7a19d8-19ad-4870-b454-a9a2dd9e6136\") "
Sep 09 03:31:59.786460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.785964    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a7a19d8-19ad-4870-b454-a9a2dd9e6136" (UID: "0a7a19d8-19ad-4870-b454-a9a2dd9e6136"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:31:59.786460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.786401    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-plugins-config-volume\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:31:59.795172 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.795143    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-kube-api-access-nxmtx" (OuterVolumeSpecName: "kube-api-access-nxmtx") pod "0a7a19d8-19ad-4870-b454-a9a2dd9e6136" (UID: "0a7a19d8-19ad-4870-b454-a9a2dd9e6136"). InnerVolumeSpecName "kube-api-access-nxmtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:31:59.887403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:31:59.887324    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxmtx\" (UniqueName: \"kubernetes.io/projected/0a7a19d8-19ad-4870-b454-a9a2dd9e6136-kube-api-access-nxmtx\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:00.193223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.193129    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a7a19d8-19ad-4870-b454-a9a2dd9e6136" containerID="f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6" exitCode=0
Sep 09 03:32:00.193223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.193200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"
Sep 09 03:32:00.193478 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.193221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw" event={"ID":"0a7a19d8-19ad-4870-b454-a9a2dd9e6136","Type":"ContainerDied","Data":"f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6"}
Sep 09 03:32:00.193478 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.193278    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw" event={"ID":"0a7a19d8-19ad-4870-b454-a9a2dd9e6136","Type":"ContainerDied","Data":"ceb5f672269576fdc3a3f09f2cfc7af2d12a623f45c037e1fee94140556b1283"}
Sep 09 03:32:00.193478 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.193301    2580 scope.go:117] "RemoveContainer" containerID="f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6"
Sep 09 03:32:00.203128 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.203113    2580 scope.go:117] "RemoveContainer" containerID="f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6"
Sep 09 03:32:00.203391 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:32:00.203369    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6\": container with ID starting with f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6 not found: ID does not exist" containerID="f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6"
Sep 09 03:32:00.203455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.203400    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6"} err="failed to get container status \"f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6\": rpc error: code = NotFound desc = could not find container \"f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6\": container with ID starting with f8e1459423ba7f2e6a959243602fd7c8afc16fa86a13863178924f20bab399c6 not found: ID does not exist"
Sep 09 03:32:00.217407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.217368    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"]
Sep 09 03:32:00.220610 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:00.220590    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ad343832-969868f8-xdpzw"]
Sep 09 03:32:01.789431 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:01.789398    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7a19d8-19ad-4870-b454-a9a2dd9e6136" path="/var/lib/kubelet/pods/0a7a19d8-19ad-4870-b454-a9a2dd9e6136/volumes"
Sep 09 03:32:02.708245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:02.708216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:32:02.708403 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:02.708260    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:32:02.720450 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:02.720427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:32:03.223142 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:03.223115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:32:06.395443 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:06.395409    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"]
Sep 09 03:32:11.687945 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:11.687912    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"]
Sep 09 03:32:11.688392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:11.688187    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" containerID="cri-o://7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd" gracePeriod=30
Sep 09 03:32:12.150687 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:12.150647    2580 patch_prober.go:28] interesting pod/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:32:12.150923 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:12.150721    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:32:16.958344 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:16.958185    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:32:17.051156 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051130    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-data\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051322 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051165    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-envoy\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051322 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-podinfo\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051322 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-credential-socket\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051322 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051280    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-socket\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istiod-ca-cert\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051363    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-certs\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051404    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt2wq\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-kube-api-access-zt2wq\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051422    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-token\") pod \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\" (UID: \"a24519fc-54ef-42c2-962f-a1cf6a0b9367\") "
Sep 09 03:32:17.051518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-data" (OuterVolumeSpecName: "istio-data") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051532    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051584    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051732    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051750    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051763    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.051813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051776    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.052131 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.051832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:32:17.053503 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.053468    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:17.053637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.053587    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:32:17.053637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.053607    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-kube-api-access-zt2wq" (OuterVolumeSpecName: "kube-api-access-zt2wq") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "kube-api-access-zt2wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:32:17.053844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.053786    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-token" (OuterVolumeSpecName: "istio-token") pod "a24519fc-54ef-42c2-962f-a1cf6a0b9367" (UID: "a24519fc-54ef-42c2-962f-a1cf6a0b9367"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:32:17.152536 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.152510    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.152536 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.152537    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt2wq\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-kube-api-access-zt2wq\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.152721 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.152551    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.152721 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.152560    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.152721 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.152567    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24519fc-54ef-42c2-962f-a1cf6a0b9367-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:17.299773 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.299740    2580 generic.go:356] "Generic (PLEG): container finished" podID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerID="7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd" exitCode=0
Sep 09 03:32:17.299969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.299795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"
Sep 09 03:32:17.299969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.299814    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" event={"ID":"a24519fc-54ef-42c2-962f-a1cf6a0b9367","Type":"ContainerDied","Data":"7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd"}
Sep 09 03:32:17.299969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.299868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2" event={"ID":"a24519fc-54ef-42c2-962f-a1cf6a0b9367","Type":"ContainerDied","Data":"894769b3e74194757cb98e017de9f91da9e64142dc4f81d3eee955c2ca5f8830"}
Sep 09 03:32:17.299969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.299890    2580 scope.go:117] "RemoveContainer" containerID="7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd"
Sep 09 03:32:17.313170 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.313151    2580 scope.go:117] "RemoveContainer" containerID="e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"
Sep 09 03:32:17.321857 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.321840    2580 scope.go:117] "RemoveContainer" containerID="7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd"
Sep 09 03:32:17.322144 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:32:17.322120    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd\": container with ID starting with 7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd not found: ID does not exist" containerID="7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd"
Sep 09 03:32:17.322209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.322154    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd"} err="failed to get container status \"7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd\": rpc error: code = NotFound desc = could not find container \"7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd\": container with ID starting with 7b4a4f6058c0975d76a93d8abd42f9683f074b8240e4906026c10fb084d937bd not found: ID does not exist"
Sep 09 03:32:17.322209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.322172    2580 scope.go:117] "RemoveContainer" containerID="e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"
Sep 09 03:32:17.322489 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:32:17.322455    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91\": container with ID starting with e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91 not found: ID does not exist" containerID="e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"
Sep 09 03:32:17.322595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.322497    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91"} err="failed to get container status \"e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91\": rpc error: code = NotFound desc = could not find container \"e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91\": container with ID starting with e822214a2f9ceb49a4e97f6c81c976a46490aba2840874e6c0ece480dcd64c91 not found: ID does not exist"
Sep 09 03:32:17.325336 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.325314    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"]
Sep 09 03:32:17.333098 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.333075    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-a2fc47-openshift-default-6b854d94f6-9z6b2"]
Sep 09 03:32:17.789866 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:17.789828    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" path="/var/lib/kubelet/pods/a24519fc-54ef-42c2-962f-a1cf6a0b9367/volumes"
Sep 09 03:32:21.415987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.415948    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p" podUID="1170ada0-488b-4f01-8add-33426181954e" containerName="main" containerID="cri-o://630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5" gracePeriod=45
Sep 09 03:32:21.668697 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.668614    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:32:21.796718 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796665    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-dshm\") pod \"1170ada0-488b-4f01-8add-33426181954e\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") "
Sep 09 03:32:21.796884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-model-cache\") pod \"1170ada0-488b-4f01-8add-33426181954e\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") "
Sep 09 03:32:21.796884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-tmp-dir\") pod \"1170ada0-488b-4f01-8add-33426181954e\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") "
Sep 09 03:32:21.796884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-home\") pod \"1170ada0-488b-4f01-8add-33426181954e\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") "
Sep 09 03:32:21.797041 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlldp\" (UniqueName: \"kubernetes.io/projected/1170ada0-488b-4f01-8add-33426181954e-kube-api-access-nlldp\") pod \"1170ada0-488b-4f01-8add-33426181954e\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") "
Sep 09 03:32:21.797041 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796957    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1170ada0-488b-4f01-8add-33426181954e-tls-certs\") pod \"1170ada0-488b-4f01-8add-33426181954e\" (UID: \"1170ada0-488b-4f01-8add-33426181954e\") "
Sep 09 03:32:21.797041 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.796978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-model-cache" (OuterVolumeSpecName: "model-cache") pod "1170ada0-488b-4f01-8add-33426181954e" (UID: "1170ada0-488b-4f01-8add-33426181954e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:21.797201 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.797113    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1170ada0-488b-4f01-8add-33426181954e" (UID: "1170ada0-488b-4f01-8add-33426181954e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:21.797263 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.797198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-home" (OuterVolumeSpecName: "home") pod "1170ada0-488b-4f01-8add-33426181954e" (UID: "1170ada0-488b-4f01-8add-33426181954e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:21.797375 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.797359    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:21.797435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.797379    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:21.797435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.797392    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:21.798812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.798786    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-dshm" (OuterVolumeSpecName: "dshm") pod "1170ada0-488b-4f01-8add-33426181954e" (UID: "1170ada0-488b-4f01-8add-33426181954e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:32:21.799247 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.799215    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1170ada0-488b-4f01-8add-33426181954e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1170ada0-488b-4f01-8add-33426181954e" (UID: "1170ada0-488b-4f01-8add-33426181954e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:32:21.799350 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.799295    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1170ada0-488b-4f01-8add-33426181954e-kube-api-access-nlldp" (OuterVolumeSpecName: "kube-api-access-nlldp") pod "1170ada0-488b-4f01-8add-33426181954e" (UID: "1170ada0-488b-4f01-8add-33426181954e"). InnerVolumeSpecName "kube-api-access-nlldp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:32:21.898997 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.898972    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlldp\" (UniqueName: \"kubernetes.io/projected/1170ada0-488b-4f01-8add-33426181954e-kube-api-access-nlldp\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:21.898997 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.898996    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1170ada0-488b-4f01-8add-33426181954e-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:21.899160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:21.899007    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1170ada0-488b-4f01-8add-33426181954e-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:32:22.333870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.333835    2580 generic.go:356] "Generic (PLEG): container finished" podID="1170ada0-488b-4f01-8add-33426181954e" containerID="630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5" exitCode=0
Sep 09 03:32:22.334045 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.333890    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"
Sep 09 03:32:22.334045 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.333926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p" event={"ID":"1170ada0-488b-4f01-8add-33426181954e","Type":"ContainerDied","Data":"630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5"}
Sep 09 03:32:22.334045 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.333969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p" event={"ID":"1170ada0-488b-4f01-8add-33426181954e","Type":"ContainerDied","Data":"db0777f5b951f9e1ff5f24922fea340fcdec242e1ec9130db7bfbd944f890ee3"}
Sep 09 03:32:22.334045 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.333990    2580 scope.go:117] "RemoveContainer" containerID="630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5"
Sep 09 03:32:22.344050 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.344035    2580 scope.go:117] "RemoveContainer" containerID="630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5"
Sep 09 03:32:22.344346 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:32:22.344329    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5\": container with ID starting with 630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5 not found: ID does not exist" containerID="630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5"
Sep 09 03:32:22.344415 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.344353    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5"} err="failed to get container status \"630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5\": rpc error: code = NotFound desc = could not find container \"630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5\": container with ID starting with 630dad9dd9a25a5429826b525cfc5ec1adb728269eebfab0d56522b73a8988f5 not found: ID does not exist"
Sep 09 03:32:22.369220 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.369196    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"]
Sep 09 03:32:22.375802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:22.375780    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-e893b71c-kserve-984bfd97c-tcj6p"]
Sep 09 03:32:23.789884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:23.789843    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1170ada0-488b-4f01-8add-33426181954e" path="/var/lib/kubelet/pods/1170ada0-488b-4f01-8add-33426181954e/volumes"
Sep 09 03:32:26.893172 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893129    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"]
Sep 09 03:32:26.893813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893793    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy"
Sep 09 03:32:26.893813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893814    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy"
Sep 09 03:32:26.893994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893826    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy"
Sep 09 03:32:26.893994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893835    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy"
Sep 09 03:32:26.893994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893851    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7a19d8-19ad-4870-b454-a9a2dd9e6136" containerName="payload-pre-processing"
Sep 09 03:32:26.893994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893859    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7a19d8-19ad-4870-b454-a9a2dd9e6136" containerName="payload-pre-processing"
Sep 09 03:32:26.893994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893876    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1170ada0-488b-4f01-8add-33426181954e" containerName="main"
Sep 09 03:32:26.893994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.893884    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1170ada0-488b-4f01-8add-33426181954e" containerName="main"
Sep 09 03:32:26.894341 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.894001    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy"
Sep 09 03:32:26.894341 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.894014    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24519fc-54ef-42c2-962f-a1cf6a0b9367" containerName="istio-proxy"
Sep 09 03:32:26.894341 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.894023    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7a19d8-19ad-4870-b454-a9a2dd9e6136" containerName="payload-pre-processing"
Sep 09 03:32:26.894341 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.894031    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1170ada0-488b-4f01-8add-33426181954e" containerName="main"
Sep 09 03:32:26.897303 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.897285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:26.900832 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.900809    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4bd6b8\"/\"kube-root-ca.crt\""
Sep 09 03:32:26.900949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.900838    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4bd6b8\"/\"openshift-service-ca.crt\""
Sep 09 03:32:26.900949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.900813    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4bd6b8\"/\"default-dockercfg-r2mf2\""
Sep 09 03:32:26.900949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.900809    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4bd6b8\"/\"test-model-4bd6b8-kserve-self-signed-certs\""
Sep 09 03:32:26.906583 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.906146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"]
Sep 09 03:32:26.954969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.954942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-dshm\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:26.955112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.954986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-home\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:26.955112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.955002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkhnd\" (UniqueName: \"kubernetes.io/projected/3665a854-b16e-473b-9a60-45172c2b09a9-kube-api-access-tkhnd\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:26.955112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.955079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-tmp-dir\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:26.955281 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.955114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3665a854-b16e-473b-9a60-45172c2b09a9-tls-certs\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:26.955281 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:26.955247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-model-cache\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056050 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-home\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056050 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkhnd\" (UniqueName: \"kubernetes.io/projected/3665a854-b16e-473b-9a60-45172c2b09a9-kube-api-access-tkhnd\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-tmp-dir\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3665a854-b16e-473b-9a60-45172c2b09a9-tls-certs\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056391 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-model-cache\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056391 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-dshm\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-tmp-dir\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-home\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.056594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.056576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-model-cache\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.058489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.058459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-dshm\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.058811 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.058794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3665a854-b16e-473b-9a60-45172c2b09a9-tls-certs\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.065586 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.065567    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkhnd\" (UniqueName: \"kubernetes.io/projected/3665a854-b16e-473b-9a60-45172c2b09a9-kube-api-access-tkhnd\") pod \"test-model-4bd6b8-kserve-74d5d8567c-bqbcz\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") " pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.208978 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.208885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:27.480201 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:27.480173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"]
Sep 09 03:32:27.481086 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:32:27.481058    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3665a854_b16e_473b_9a60_45172c2b09a9.slice/crio-fa01c328615a410283a720189fa426b5941ef712c6c1af2aab8dd586121edef8 WatchSource:0}: Error finding container fa01c328615a410283a720189fa426b5941ef712c6c1af2aab8dd586121edef8: Status 404 returned error can't find the container with id fa01c328615a410283a720189fa426b5941ef712c6c1af2aab8dd586121edef8
Sep 09 03:32:28.374457 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:28.374420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz" event={"ID":"3665a854-b16e-473b-9a60-45172c2b09a9","Type":"ContainerStarted","Data":"4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff"}
Sep 09 03:32:28.374457 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:28.374464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz" event={"ID":"3665a854-b16e-473b-9a60-45172c2b09a9","Type":"ContainerStarted","Data":"fa01c328615a410283a720189fa426b5941ef712c6c1af2aab8dd586121edef8"}
Sep 09 03:32:28.396455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:28.396405    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz" podStartSLOduration=2.396388572 podStartE2EDuration="2.396388572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:32:28.393957348 +0000 UTC m=+1159.093721301" watchObservedRunningTime="2026-09-09 03:32:28.396388572 +0000 UTC m=+1159.096152533"
Sep 09 03:32:37.209664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:37.209588    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:37.209664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:37.209632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:37.221754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:37.221734    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:32:37.444776 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:32:37.444752    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:33:41.422411 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:41.422376    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"]
Sep 09 03:33:52.174258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.174224    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"]
Sep 09 03:33:52.174648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.174470    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" podUID="0a9a1044-a84b-47b3-b187-0f7831119ed6" containerName="maas-api" containerID="cri-o://4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f" gracePeriod=30
Sep 09 03:33:52.198460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.198428    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"]
Sep 09 03:33:52.201353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.201330    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-40a1c9-29815410-2mdnf"]
Sep 09 03:33:52.421100 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.421078    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:33:52.544772 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.544743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82d62\" (UniqueName: \"kubernetes.io/projected/0a9a1044-a84b-47b3-b187-0f7831119ed6-kube-api-access-82d62\") pod \"0a9a1044-a84b-47b3-b187-0f7831119ed6\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") "
Sep 09 03:33:52.544951 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.544830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a9a1044-a84b-47b3-b187-0f7831119ed6-openshift-service-ca\") pod \"0a9a1044-a84b-47b3-b187-0f7831119ed6\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") "
Sep 09 03:33:52.544951 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.544925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-metrics-certs\") pod \"0a9a1044-a84b-47b3-b187-0f7831119ed6\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") "
Sep 09 03:33:52.545090 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.544955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls\") pod \"0a9a1044-a84b-47b3-b187-0f7831119ed6\" (UID: \"0a9a1044-a84b-47b3-b187-0f7831119ed6\") "
Sep 09 03:33:52.545306 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.545260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a9a1044-a84b-47b3-b187-0f7831119ed6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a9a1044-a84b-47b3-b187-0f7831119ed6" (UID: "0a9a1044-a84b-47b3-b187-0f7831119ed6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:33:52.547430 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.547405    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0a9a1044-a84b-47b3-b187-0f7831119ed6" (UID: "0a9a1044-a84b-47b3-b187-0f7831119ed6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:33:52.547521 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.547436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9a1044-a84b-47b3-b187-0f7831119ed6-kube-api-access-82d62" (OuterVolumeSpecName: "kube-api-access-82d62") pod "0a9a1044-a84b-47b3-b187-0f7831119ed6" (UID: "0a9a1044-a84b-47b3-b187-0f7831119ed6"). InnerVolumeSpecName "kube-api-access-82d62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:52.547600 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.547577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a9a1044-a84b-47b3-b187-0f7831119ed6" (UID: "0a9a1044-a84b-47b3-b187-0f7831119ed6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:33:52.646315 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.646279    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82d62\" (UniqueName: \"kubernetes.io/projected/0a9a1044-a84b-47b3-b187-0f7831119ed6-kube-api-access-82d62\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.646315 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.646317    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a9a1044-a84b-47b3-b187-0f7831119ed6-openshift-service-ca\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.646498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.646334    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-metrics-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.646498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.646348    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a9a1044-a84b-47b3-b187-0f7831119ed6-maas-api-tls\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:52.906160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.906078    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a9a1044-a84b-47b3-b187-0f7831119ed6" containerID="4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f" exitCode=0
Sep 09 03:33:52.906160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.906122    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"
Sep 09 03:33:52.906347 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.906161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" event={"ID":"0a9a1044-a84b-47b3-b187-0f7831119ed6","Type":"ContainerDied","Data":"4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f"}
Sep 09 03:33:52.906347 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.906238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9" event={"ID":"0a9a1044-a84b-47b3-b187-0f7831119ed6","Type":"ContainerDied","Data":"f2fb71872e6d4412ff6179268dc36c7c17a4cf5c667c04e7c60e183bbdd8a687"}
Sep 09 03:33:52.906347 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.906260    2580 scope.go:117] "RemoveContainer" containerID="4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f"
Sep 09 03:33:52.916493 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.916471    2580 scope.go:117] "RemoveContainer" containerID="4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f"
Sep 09 03:33:52.916797 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:33:52.916778    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f\": container with ID starting with 4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f not found: ID does not exist" containerID="4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f"
Sep 09 03:33:52.916858 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.916804    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f"} err="failed to get container status \"4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f\": rpc error: code = NotFound desc = could not find container \"4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f\": container with ID starting with 4ae6c8f00d2b86f3d750c7cd1636e5f25b34d2b197bbe71b61b5e665ea20f54f not found: ID does not exist"
Sep 09 03:33:52.941060 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.941032    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"]
Sep 09 03:33:52.963418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:52.963397    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-40a1c9-565bff4d66-tnsp9"]
Sep 09 03:33:53.789480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:53.789445    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a9a1044-a84b-47b3-b187-0f7831119ed6" path="/var/lib/kubelet/pods/0a9a1044-a84b-47b3-b187-0f7831119ed6/volumes"
Sep 09 03:33:53.789864 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:53.789742    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e9367f0-913e-458b-ac17-d075d5c3ac48" path="/var/lib/kubelet/pods/7e9367f0-913e-458b-ac17-d075d5c3ac48/volumes"
Sep 09 03:33:56.442551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.442512    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz" podUID="3665a854-b16e-473b-9a60-45172c2b09a9" containerName="main" containerID="cri-o://4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff" gracePeriod=45
Sep 09 03:33:56.694455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.694402    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:33:56.785594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785567    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-dshm\") pod \"3665a854-b16e-473b-9a60-45172c2b09a9\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") "
Sep 09 03:33:56.785785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785601    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-model-cache\") pod \"3665a854-b16e-473b-9a60-45172c2b09a9\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") "
Sep 09 03:33:56.785785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785622    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3665a854-b16e-473b-9a60-45172c2b09a9-tls-certs\") pod \"3665a854-b16e-473b-9a60-45172c2b09a9\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") "
Sep 09 03:33:56.785785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-tmp-dir\") pod \"3665a854-b16e-473b-9a60-45172c2b09a9\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") "
Sep 09 03:33:56.785785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785730    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkhnd\" (UniqueName: \"kubernetes.io/projected/3665a854-b16e-473b-9a60-45172c2b09a9-kube-api-access-tkhnd\") pod \"3665a854-b16e-473b-9a60-45172c2b09a9\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") "
Sep 09 03:33:56.785785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-home\") pod \"3665a854-b16e-473b-9a60-45172c2b09a9\" (UID: \"3665a854-b16e-473b-9a60-45172c2b09a9\") "
Sep 09 03:33:56.786050 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.785898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "3665a854-b16e-473b-9a60-45172c2b09a9" (UID: "3665a854-b16e-473b-9a60-45172c2b09a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:56.786108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.786041    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-home" (OuterVolumeSpecName: "home") pod "3665a854-b16e-473b-9a60-45172c2b09a9" (UID: "3665a854-b16e-473b-9a60-45172c2b09a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:56.786108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.786057    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3665a854-b16e-473b-9a60-45172c2b09a9" (UID: "3665a854-b16e-473b-9a60-45172c2b09a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:56.786233 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.786140    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:56.786233 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.786159    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:56.786233 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.786170    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:56.788033 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.788009    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3665a854-b16e-473b-9a60-45172c2b09a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3665a854-b16e-473b-9a60-45172c2b09a9" (UID: "3665a854-b16e-473b-9a60-45172c2b09a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:33:56.788106 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.788089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-dshm" (OuterVolumeSpecName: "dshm") pod "3665a854-b16e-473b-9a60-45172c2b09a9" (UID: "3665a854-b16e-473b-9a60-45172c2b09a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:33:56.788164 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.788107    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3665a854-b16e-473b-9a60-45172c2b09a9-kube-api-access-tkhnd" (OuterVolumeSpecName: "kube-api-access-tkhnd") pod "3665a854-b16e-473b-9a60-45172c2b09a9" (UID: "3665a854-b16e-473b-9a60-45172c2b09a9"). InnerVolumeSpecName "kube-api-access-tkhnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:33:56.887889 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.887860    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3665a854-b16e-473b-9a60-45172c2b09a9-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:56.887889 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.887887    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3665a854-b16e-473b-9a60-45172c2b09a9-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:56.888093 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.887902    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkhnd\" (UniqueName: \"kubernetes.io/projected/3665a854-b16e-473b-9a60-45172c2b09a9-kube-api-access-tkhnd\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:33:56.932961 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.932929    2580 generic.go:356] "Generic (PLEG): container finished" podID="3665a854-b16e-473b-9a60-45172c2b09a9" containerID="4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff" exitCode=0
Sep 09 03:33:56.933108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.932987    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"
Sep 09 03:33:56.933108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.933017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz" event={"ID":"3665a854-b16e-473b-9a60-45172c2b09a9","Type":"ContainerDied","Data":"4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff"}
Sep 09 03:33:56.933108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.933071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz" event={"ID":"3665a854-b16e-473b-9a60-45172c2b09a9","Type":"ContainerDied","Data":"fa01c328615a410283a720189fa426b5941ef712c6c1af2aab8dd586121edef8"}
Sep 09 03:33:56.933108 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.933095    2580 scope.go:117] "RemoveContainer" containerID="4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff"
Sep 09 03:33:56.944245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.944228    2580 scope.go:117] "RemoveContainer" containerID="4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff"
Sep 09 03:33:56.944569 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:33:56.944523    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff\": container with ID starting with 4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff not found: ID does not exist" containerID="4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff"
Sep 09 03:33:56.944569 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.944550    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff"} err="failed to get container status \"4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff\": rpc error: code = NotFound desc = could not find container \"4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff\": container with ID starting with 4b973dda8b817e81557fcda90dab08a9e79ba15e3ba09c9dd6998ac72d5280ff not found: ID does not exist"
Sep 09 03:33:56.959006 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.958985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"]
Sep 09 03:33:56.963204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:56.963181    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4bd6b8/test-model-4bd6b8-kserve-74d5d8567c-bqbcz"]
Sep 09 03:33:57.789510 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:57.789480    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3665a854-b16e-473b-9a60-45172c2b09a9" path="/var/lib/kubelet/pods/3665a854-b16e-473b-9a60-45172c2b09a9/volumes"
Sep 09 03:33:59.380358 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:59.380329    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"]
Sep 09 03:33:59.380743 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:33:59.380542    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy" containerID="cri-o://34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9" gracePeriod=30
Sep 09 03:34:04.638386 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.638358    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:34:04.759489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759462    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-data\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759632 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759560    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-token\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759632 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759609    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-envoy\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759632 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759627    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-credential-socket\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759652    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-certs\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759708    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-socket\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-podinfo\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-data" (OuterVolumeSpecName: "istio-data") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:04.759878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759776    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj75d\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-kube-api-access-fj75d\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.759878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759825    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istiod-ca-cert\") pod \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\" (UID: \"57f8ad20-0d73-47ef-ad8e-41da0c987e47\") "
Sep 09 03:34:04.760204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.759926    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:04.760204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760005    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:04.760204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:04.760359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760262    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.760359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760284    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.760359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760300    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.760359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760315    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.760527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.760419    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:04.762073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.762036    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:04.762073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.762050    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-kube-api-access-fj75d" (OuterVolumeSpecName: "kube-api-access-fj75d") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "kube-api-access-fj75d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:04.762397 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.762379    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:34:04.762451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.762392    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-token" (OuterVolumeSpecName: "istio-token") pod "57f8ad20-0d73-47ef-ad8e-41da0c987e47" (UID: "57f8ad20-0d73-47ef-ad8e-41da0c987e47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:04.861220 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.861195    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.861220 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.861219    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.861369 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.861232    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.861369 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.861247    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj75d\" (UniqueName: \"kubernetes.io/projected/57f8ad20-0d73-47ef-ad8e-41da0c987e47-kube-api-access-fj75d\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.861369 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.861260    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57f8ad20-0d73-47ef-ad8e-41da0c987e47-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:04.982282 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.982260    2580 generic.go:356] "Generic (PLEG): container finished" podID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerID="34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9" exitCode=0
Sep 09 03:34:04.982392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.982306    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"
Sep 09 03:34:04.982392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.982346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" event={"ID":"57f8ad20-0d73-47ef-ad8e-41da0c987e47","Type":"ContainerDied","Data":"34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9"}
Sep 09 03:34:04.982465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.982395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c" event={"ID":"57f8ad20-0d73-47ef-ad8e-41da0c987e47","Type":"ContainerDied","Data":"6cd3b9af64d0a7f0163cc776323e0dcbbb02822baa69884fa0cf0b6c5cdafa89"}
Sep 09 03:34:04.982465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.982420    2580 scope.go:117] "RemoveContainer" containerID="34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9"
Sep 09 03:34:04.994455 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:04.994437    2580 scope.go:117] "RemoveContainer" containerID="4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"
Sep 09 03:34:05.003046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.003027    2580 scope.go:117] "RemoveContainer" containerID="34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9"
Sep 09 03:34:05.003292 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:34:05.003273    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9\": container with ID starting with 34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9 not found: ID does not exist" containerID="34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9"
Sep 09 03:34:05.003366 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.003305    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9"} err="failed to get container status \"34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9\": rpc error: code = NotFound desc = could not find container \"34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9\": container with ID starting with 34ba85ab358356bb4ececa94d3b08ad06ec9c96f29aae0cf18f8d1ba9827ccd9 not found: ID does not exist"
Sep 09 03:34:05.003366 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.003329    2580 scope.go:117] "RemoveContainer" containerID="4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"
Sep 09 03:34:05.003598 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:34:05.003582    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723\": container with ID starting with 4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723 not found: ID does not exist" containerID="4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"
Sep 09 03:34:05.003636 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.003605    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723"} err="failed to get container status \"4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723\": rpc error: code = NotFound desc = could not find container \"4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723\": container with ID starting with 4d74d45e29795bfd733a2212e5072500dccf66d69453f560bc7c82616e553723 not found: ID does not exist"
Sep 09 03:34:05.015551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.015503    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"]
Sep 09 03:34:05.027566 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.027546    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-40a1c9-openshift-default-6456f7b598-6lg8c"]
Sep 09 03:34:05.789934 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:05.789902    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" path="/var/lib/kubelet/pods/57f8ad20-0d73-47ef-ad8e-41da0c987e47/volumes"
Sep 09 03:34:18.742235 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:18.742201    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"]
Sep 09 03:34:18.742828 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:18.742587    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy" containerID="cri-o://69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a" gracePeriod=30
Sep 09 03:34:19.734413 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734372    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"]
Sep 09 03:34:19.734815 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734801    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy"
Sep 09 03:34:19.734815 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734816    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734827    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a9a1044-a84b-47b3-b187-0f7831119ed6" containerName="maas-api"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734832    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a9a1044-a84b-47b3-b187-0f7831119ed6" containerName="maas-api"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734844    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734849    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734865    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3665a854-b16e-473b-9a60-45172c2b09a9" containerName="main"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734871    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3665a854-b16e-473b-9a60-45172c2b09a9" containerName="main"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734931    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a9a1044-a84b-47b3-b187-0f7831119ed6" containerName="maas-api"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734939    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734944    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f8ad20-0d73-47ef-ad8e-41da0c987e47" containerName="istio-proxy"
Sep 09 03:34:19.734963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.734953    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3665a854-b16e-473b-9a60-45172c2b09a9" containerName="main"
Sep 09 03:34:19.737587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.737570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.740111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.740090    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-5d8e85-openshift-default-dockercfg-7f5v7\""
Sep 09 03:34:19.783069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.783047    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"]
Sep 09 03:34:19.793367 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-istio-token\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c88bm\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-kube-api-access-c88bm\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf0e463-819e-4776-9224-8b05787dd61c-istio-podinfo\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-credential-socket\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-data\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-certs\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf0e463-819e-4776-9224-8b05787dd61c-istiod-ca-cert\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-envoy\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.793754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.793653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-socket\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.894917 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.894879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf0e463-819e-4776-9224-8b05787dd61c-istiod-ca-cert\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.894917 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.894920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-envoy\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.894964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-socket\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-istio-token\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c88bm\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-kube-api-access-c88bm\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895326 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf0e463-819e-4776-9224-8b05787dd61c-istio-podinfo\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895326 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-credential-socket\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-data\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-socket\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895513 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-certs\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895632 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-credential-socket\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895750 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-data\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895750 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf0e463-819e-4776-9224-8b05787dd61c-istiod-ca-cert\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.895862 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.895842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-certs\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.897452 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.897436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-envoy\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.897827 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.897806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf0e463-819e-4776-9224-8b05787dd61c-istio-podinfo\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.915766 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.915734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-istio-token\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:19.915950 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:19.915933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c88bm\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-kube-api-access-c88bm\") pod \"e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") " pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:20.048294 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:20.048225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:20.335813 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:20.335783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"]
Sep 09 03:34:20.335914 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:34:20.335834    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf0e463_819e_4776_9224_8b05787dd61c.slice/crio-99efb4f01c57712c2d2c02ac912220a7d136fc37d0c712abff1e0d7b55a13123 WatchSource:0}: Error finding container 99efb4f01c57712c2d2c02ac912220a7d136fc37d0c712abff1e0d7b55a13123: Status 404 returned error can't find the container with id 99efb4f01c57712c2d2c02ac912220a7d136fc37d0c712abff1e0d7b55a13123
Sep 09 03:34:20.338174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:20.338136    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:34:20.338263 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:20.338251    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:34:20.338310 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:20.338293    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:34:21.079264 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:21.079223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" event={"ID":"1cf0e463-819e-4776-9224-8b05787dd61c","Type":"ContainerStarted","Data":"6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"}
Sep 09 03:34:21.079877 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:21.079281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" event={"ID":"1cf0e463-819e-4776-9224-8b05787dd61c","Type":"ContainerStarted","Data":"99efb4f01c57712c2d2c02ac912220a7d136fc37d0c712abff1e0d7b55a13123"}
Sep 09 03:34:21.108394 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:21.108354    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podStartSLOduration=2.108340358 podStartE2EDuration="2.108340358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:34:21.107435868 +0000 UTC m=+1271.807199836" watchObservedRunningTime="2026-09-09 03:34:21.108340358 +0000 UTC m=+1271.808104321"
Sep 09 03:34:22.049242 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:22.049215    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:22.054273 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:22.054246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:22.085920 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:22.085895    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:22.086645 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:22.086628    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:23.982321 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:23.982296    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:34:24.033524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033437    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istiod-ca-cert\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.033524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033507    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-token\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.033793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-envoy\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.033793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-podinfo\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.033793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-credential-socket\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.033793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-data\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.033793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033767    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-certs\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.034065 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krj4t\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-kube-api-access-krj4t\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.034065 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-socket\") pod \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\" (UID: \"e1930e3c-390d-44bd-87d7-03ead6aa48e7\") "
Sep 09 03:34:24.034065 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033906    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:34:24.034065 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.033998    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:24.034286 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.034232    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.034286 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.034239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:24.034286 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.034253    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.034708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.034645    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:24.034882 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.034860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:24.036623 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.036596    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:34:24.036878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.036847    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-kube-api-access-krj4t" (OuterVolumeSpecName: "kube-api-access-krj4t") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "kube-api-access-krj4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:24.036968 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.036883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:34:24.037774 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.037758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1930e3c-390d-44bd-87d7-03ead6aa48e7" (UID: "e1930e3c-390d-44bd-87d7-03ead6aa48e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:34:24.098866 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.098836    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerID="69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a" exitCode=0
Sep 09 03:34:24.098987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.098885    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"
Sep 09 03:34:24.098987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.098914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" event={"ID":"e1930e3c-390d-44bd-87d7-03ead6aa48e7","Type":"ContainerDied","Data":"69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a"}
Sep 09 03:34:24.098987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.098973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9" event={"ID":"e1930e3c-390d-44bd-87d7-03ead6aa48e7","Type":"ContainerDied","Data":"523c5b216624e2888f95c9bfee46d932f45ec86b68da29da6262d33eb64225ca"}
Sep 09 03:34:24.099111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.099004    2580 scope.go:117] "RemoveContainer" containerID="69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a"
Sep 09 03:34:24.109418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.109403    2580 scope.go:117] "RemoveContainer" containerID="0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"
Sep 09 03:34:24.117914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.117898    2580 scope.go:117] "RemoveContainer" containerID="69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a"
Sep 09 03:34:24.118174 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:34:24.118155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a\": container with ID starting with 69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a not found: ID does not exist" containerID="69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a"
Sep 09 03:34:24.118230 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.118191    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a"} err="failed to get container status \"69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a\": rpc error: code = NotFound desc = could not find container \"69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a\": container with ID starting with 69bf6aa312d13455dfbd0444701e7a70702a65e4e9125fdc4758e9280b33816a not found: ID does not exist"
Sep 09 03:34:24.118230 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.118208    2580 scope.go:117] "RemoveContainer" containerID="0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"
Sep 09 03:34:24.118467 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:34:24.118447    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715\": container with ID starting with 0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715 not found: ID does not exist" containerID="0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"
Sep 09 03:34:24.118525 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.118477    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715"} err="failed to get container status \"0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715\": rpc error: code = NotFound desc = could not find container \"0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715\": container with ID starting with 0d303c05d87b47de92f42d21140db9ed99ddee97eab598a0765817fc4b2a5715 not found: ID does not exist"
Sep 09 03:34:24.123914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.123896    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"]
Sep 09 03:34:24.132996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.132974    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4bd6b8-openshift-default-7bddd9fddd-7lmn9"]
Sep 09 03:34:24.135608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135585    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.135735 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135698    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.135830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135809    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.135931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135833    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krj4t\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-kube-api-access-krj4t\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.135931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135849    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.135931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135860    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:24.135931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:24.135868    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1930e3c-390d-44bd-87d7-03ead6aa48e7-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:34:25.789319 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:25.789282    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" path="/var/lib/kubelet/pods/e1930e3c-390d-44bd-87d7-03ead6aa48e7/volumes"
Sep 09 03:34:37.317021 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:37.316977    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:46536->10.134.0.63:15021: read: connection reset by peer" start-of-body=
Sep 09 03:34:37.317447 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:37.317069    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:46536->10.134.0.63:15021: read: connection reset by peer"
Sep 09 03:34:38.195550 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:38.195519    2580 generic.go:356] "Generic (PLEG): container finished" podID="1cf0e463-819e-4776-9224-8b05787dd61c" containerID="6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1" exitCode=0
Sep 09 03:34:38.195753 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:38.195606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" event={"ID":"1cf0e463-819e-4776-9224-8b05787dd61c","Type":"ContainerDied","Data":"6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"}
Sep 09 03:34:38.196079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:38.196065    2580 scope.go:117] "RemoveContainer" containerID="6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"
Sep 09 03:34:38.197141 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:38.197109    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:34:38.197257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:38.197191    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:34:38.197257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:38.197239    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:34:39.203143 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:39.203114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" event={"ID":"1cf0e463-819e-4776-9224-8b05787dd61c","Type":"ContainerStarted","Data":"4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39"}
Sep 09 03:34:40.048608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:40.048570    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:40.048938 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:40.048906    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 09 03:34:40.049050 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:40.048966    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 09 03:34:41.049303 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:41.049266    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 09 03:34:41.049926 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:41.049893    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 09 03:34:42.049424 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:42.049385    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 09 03:34:42.050066 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:42.050033    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 09 03:34:43.049406 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:43.049361    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 09 03:34:43.049614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:43.049428    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 09 03:34:44.048718 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:44.048651    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 09 03:34:44.048934 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:44.048746    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 09 03:34:45.051849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:45.051822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:45.052293 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:45.052277    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:34:45.053044 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:34:45.053024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:35:02.942353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.942318    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"]
Sep 09 03:35:02.942954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.942933    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy"
Sep 09 03:35:02.943041 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.942957    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy"
Sep 09 03:35:02.943110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.943096    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy"
Sep 09 03:35:02.943165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.943116    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy"
Sep 09 03:35:02.945761 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.945741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.949752 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:35:02.949721    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-shared-a-w2-eca017-openshift-default-dockercfg-xclvd\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-eca017-openshift-default-dockercfg-xclvd\"" type="*v1.Secret"
Sep 09 03:35:02.952486 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:35:02.952458    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990512 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-data\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990599 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-istio-token\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990599 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-socket\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990599 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee436-2a31-4741-bc26-693703b1fa69-istiod-ca-cert\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-credential-socket\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-envoy\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-certs\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990739 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sr9f\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-kube-api-access-6sr9f\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.990869 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.990780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49ee436-2a31-4741-bc26-693703b1fa69-istio-podinfo\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:02.996278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:02.996257    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"]
Sep 09 03:35:03.092087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-socket\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee436-2a31-4741-bc26-693703b1fa69-istiod-ca-cert\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-credential-socket\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-envoy\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-certs\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sr9f\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-kube-api-access-6sr9f\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49ee436-2a31-4741-bc26-693703b1fa69-istio-podinfo\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-data\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-socket\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-istio-token\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-credential-socket\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-data\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092976 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-certs\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.092976 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.092971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee436-2a31-4741-bc26-693703b1fa69-istiod-ca-cert\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.094790 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.094769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-envoy\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.094935 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.094918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49ee436-2a31-4741-bc26-693703b1fa69-istio-podinfo\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.115848 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.115827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-istio-token\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.116876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.116857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sr9f\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-kube-api-access-6sr9f\") pod \"e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") " pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:03.845533 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.845502    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-eca017-openshift-default-dockercfg-xclvd\""
Sep 09 03:35:03.846535 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:03.846519    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:04.136305 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.136277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"]
Sep 09 03:35:04.137865 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:35:04.137827    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49ee436_2a31_4741_bc26_693703b1fa69.slice/crio-ce3878e54c3e6a19eb5608c9289a190b76b91b0075a0739a637e8169cbc288da WatchSource:0}: Error finding container ce3878e54c3e6a19eb5608c9289a190b76b91b0075a0739a637e8169cbc288da: Status 404 returned error can't find the container with id ce3878e54c3e6a19eb5608c9289a190b76b91b0075a0739a637e8169cbc288da
Sep 09 03:35:04.140362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.140316    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:04.140449 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.140403    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:04.140493 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.140446    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:04.367094 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.367059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" event={"ID":"e49ee436-2a31-4741-bc26-693703b1fa69","Type":"ContainerStarted","Data":"ec66287ca832353df3c9cbffd96a8fc62d1def8d5375b06e2a464bf550e19360"}
Sep 09 03:35:04.367254 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.367129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" event={"ID":"e49ee436-2a31-4741-bc26-693703b1fa69","Type":"ContainerStarted","Data":"ce3878e54c3e6a19eb5608c9289a190b76b91b0075a0739a637e8169cbc288da"}
Sep 09 03:35:04.390284 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.390192    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podStartSLOduration=2.390160895 podStartE2EDuration="2.390160895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:04.388114496 +0000 UTC m=+1315.087878484" watchObservedRunningTime="2026-09-09 03:35:04.390160895 +0000 UTC m=+1315.089924859"
Sep 09 03:35:04.846859 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:04.846829    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:05.851545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:05.851517    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:06.378768 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:06.378739    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:06.379787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:06.379770    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:22.419808 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:22.419771    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 03:35:22.420268 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:22.419829    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 03:35:23.494874 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:23.494839    2580 generic.go:356] "Generic (PLEG): container finished" podID="e49ee436-2a31-4741-bc26-693703b1fa69" containerID="ec66287ca832353df3c9cbffd96a8fc62d1def8d5375b06e2a464bf550e19360" exitCode=0
Sep 09 03:35:23.495265 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:23.494917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" event={"ID":"e49ee436-2a31-4741-bc26-693703b1fa69","Type":"ContainerDied","Data":"ec66287ca832353df3c9cbffd96a8fc62d1def8d5375b06e2a464bf550e19360"}
Sep 09 03:35:23.495370 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:23.495353    2580 scope.go:117] "RemoveContainer" containerID="ec66287ca832353df3c9cbffd96a8fc62d1def8d5375b06e2a464bf550e19360"
Sep 09 03:35:23.496446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:23.496419    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:23.496521 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:23.496507    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:23.496557 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:23.496540    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:24.502379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:24.502342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" event={"ID":"e49ee436-2a31-4741-bc26-693703b1fa69","Type":"ContainerStarted","Data":"38f2bf17c1d3c850992d3fffc95c8ee99eb4508ea5d84e24455df313d1d3df0d"}
Sep 09 03:35:24.848162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:24.847648    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:24.848162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:24.847797    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 09 03:35:24.848162 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:24.847842    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 09 03:35:25.103450 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.103378    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"]
Sep 09 03:35:25.103890 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.103872    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy"
Sep 09 03:35:25.104034 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.104019    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1930e3c-390d-44bd-87d7-03ead6aa48e7" containerName="istio-proxy"
Sep 09 03:35:25.107720 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.107700    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.110424 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.110404    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"kube-root-ca.crt\""
Sep 09 03:35:25.111370 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.111352    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"default-dockercfg-r99j8\""
Sep 09 03:35:25.111566 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.111545    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"openshift-service-ca.crt\""
Sep 09 03:35:25.111687 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.111570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"auth-test-model-5d8e85-kserve-self-signed-certs\""
Sep 09 03:35:25.123184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.123164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"]
Sep 09 03:35:25.201797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.201604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tmp-dir\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.202087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.202063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tls-certs\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.208435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.202175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-dshm\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.208435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.202276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-home\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.208435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.202328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2snr\" (UniqueName: \"kubernetes.io/projected/83ecedd9-008e-4375-bda9-d1766b7e9b4e-kube-api-access-j2snr\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.208435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.202439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-model-cache\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303339 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-model-cache\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tmp-dir\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tls-certs\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-dshm\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-home\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303531 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2snr\" (UniqueName: \"kubernetes.io/projected/83ecedd9-008e-4375-bda9-d1766b7e9b4e-kube-api-access-j2snr\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303886 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-model-cache\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.303958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-home\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.304001 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.303983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tmp-dir\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.305870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.305840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-dshm\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.306018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.305999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tls-certs\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.313003 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.312971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2snr\" (UniqueName: \"kubernetes.io/projected/83ecedd9-008e-4375-bda9-d1766b7e9b4e-kube-api-access-j2snr\") pod \"auth-test-model-5d8e85-kserve-7785689fdc-rjjhk\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.417200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.417119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:25.721558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.721526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"]
Sep 09 03:35:25.722217 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:35:25.722180    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83ecedd9_008e_4375_bda9_d1766b7e9b4e.slice/crio-89403e40884bcf904e35fdf90c5758e626ba7988262dac927f41946b835ab0cf WatchSource:0}: Error finding container 89403e40884bcf904e35fdf90c5758e626ba7988262dac927f41946b835ab0cf: Status 404 returned error can't find the container with id 89403e40884bcf904e35fdf90c5758e626ba7988262dac927f41946b835ab0cf
Sep 09 03:35:25.847889 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.847849    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 09 03:35:25.848055 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:25.847914    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 09 03:35:26.519742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:26.519688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk" event={"ID":"83ecedd9-008e-4375-bda9-d1766b7e9b4e","Type":"ContainerStarted","Data":"ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd"}
Sep 09 03:35:26.519742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:26.519754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk" event={"ID":"83ecedd9-008e-4375-bda9-d1766b7e9b4e","Type":"ContainerStarted","Data":"89403e40884bcf904e35fdf90c5758e626ba7988262dac927f41946b835ab0cf"}
Sep 09 03:35:26.539268 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:26.539210    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk" podStartSLOduration=1.539192044 podStartE2EDuration="1.539192044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:26.537304463 +0000 UTC m=+1337.237068451" watchObservedRunningTime="2026-09-09 03:35:26.539192044 +0000 UTC m=+1337.238956007"
Sep 09 03:35:26.847862 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:26.847768    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 09 03:35:26.847862 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:26.847845    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 09 03:35:27.847715 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:27.847653    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 09 03:35:27.847899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:27.847741    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 09 03:35:28.847747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:28.847703    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 09 03:35:28.847956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:28.847770    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 09 03:35:29.850791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:29.850757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:29.851270 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:29.851127    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:29.851906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:29.851890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:35:32.034232 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.034153    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"]
Sep 09 03:35:32.037894 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.037878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.040851 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.040830    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-0d0cae-openshift-default-dockercfg-bgmgv\""
Sep 09 03:35:32.052335 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.052314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"]
Sep 09 03:35:32.173253 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-podinfo\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173253 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-data\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-socket\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qhr2\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-kube-api-access-8qhr2\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-envoy\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-certs\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istiod-ca-cert\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-credential-socket\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.173488 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.173471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-token\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-podinfo\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-data\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-socket\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qhr2\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-kube-api-access-8qhr2\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-envoy\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-certs\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274396 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istiod-ca-cert\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-credential-socket\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-token\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-data\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274917 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-certs\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-socket\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.274982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.274968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-credential-socket\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.275251 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.275233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istiod-ca-cert\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.276770 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.276747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-envoy\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.277200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.277182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-podinfo\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.288046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.286997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qhr2\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-kube-api-access-8qhr2\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.288386 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.288359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-token\") pod \"e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") " pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.351274 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.351250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:32.647318 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:35:32.647281    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41d7cdc_57d2_427d_ba6d_4260c30a3379.slice/crio-fc80f37a16cbe4d76ef8dcbb94131c92785e059a4095d7a8614b8f8c986575d7 WatchSource:0}: Error finding container fc80f37a16cbe4d76ef8dcbb94131c92785e059a4095d7a8614b8f8c986575d7: Status 404 returned error can't find the container with id fc80f37a16cbe4d76ef8dcbb94131c92785e059a4095d7a8614b8f8c986575d7
Sep 09 03:35:32.653271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.653176    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:32.653941 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.653905    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:32.654615 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.654586    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:32.657572 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:32.657551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"]
Sep 09 03:35:33.576443 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:33.576407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" event={"ID":"c41d7cdc-57d2-427d-ba6d-4260c30a3379","Type":"ContainerStarted","Data":"e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"}
Sep 09 03:35:33.576850 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:33.576464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" event={"ID":"c41d7cdc-57d2-427d-ba6d-4260c30a3379","Type":"ContainerStarted","Data":"fc80f37a16cbe4d76ef8dcbb94131c92785e059a4095d7a8614b8f8c986575d7"}
Sep 09 03:35:33.600642 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:33.600596    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podStartSLOduration=1.600580707 podStartE2EDuration="1.600580707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:33.59907553 +0000 UTC m=+1344.298839491" watchObservedRunningTime="2026-09-09 03:35:33.600580707 +0000 UTC m=+1344.300344669"
Sep 09 03:35:34.352460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:34.352427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:34.358258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:34.358234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:34.582255 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:34.582229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:34.583048 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:34.583031    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:35.417633 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:35.417604    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:35.417822 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:35.417645    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:35.432180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:35.432154    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:35.599100 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:35.599070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:35:51.493952 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.493916    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"]
Sep 09 03:35:51.498072 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.498051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.503978 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.503955    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"kube-root-ca.crt\""
Sep 09 03:35:51.505095 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.505069    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"openshift-service-ca.crt\""
Sep 09 03:35:51.505223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.505073    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"auth-test-model-c34148-kserve-self-signed-certs\""
Sep 09 03:35:51.505223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.505131    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"default-dockercfg-gwznt\""
Sep 09 03:35:51.514249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.514227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"]
Sep 09 03:35:51.564321 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.564289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-home\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.564497 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.564338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-tmp-dir\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.564497 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.564388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-model-cache\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.564497 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.564446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6cb293-76b0-4f53-8d51-e38862d793e6-tls-certs\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.564497 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.564467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwcck\" (UniqueName: \"kubernetes.io/projected/fb6cb293-76b0-4f53-8d51-e38862d793e6-kube-api-access-hwcck\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.564634 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.564558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-dshm\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.665521 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.665486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-dshm\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.665729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.665560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-home\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.665729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.665617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-tmp-dir\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.665729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.665645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-model-cache\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.665729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.665707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6cb293-76b0-4f53-8d51-e38862d793e6-tls-certs\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.665729 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.665727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwcck\" (UniqueName: \"kubernetes.io/projected/fb6cb293-76b0-4f53-8d51-e38862d793e6-kube-api-access-hwcck\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.666063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.666038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-home\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.666249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.666104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-tmp-dir\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.666333 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.666132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-model-cache\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.668117 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.668087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-dshm\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.668340 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.668323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6cb293-76b0-4f53-8d51-e38862d793e6-tls-certs\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.675362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.675342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwcck\" (UniqueName: \"kubernetes.io/projected/fb6cb293-76b0-4f53-8d51-e38862d793e6-kube-api-access-hwcck\") pod \"auth-test-model-c34148-kserve-744c6dcf87-4sh2w\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") " pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:51.810451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:51.810374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:35:52.090622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:52.090592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"]
Sep 09 03:35:52.092297 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:35:52.092273    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb6cb293_76b0_4f53_8d51_e38862d793e6.slice/crio-7d231e6adfc1c9652b86c547b9b0d4be1e622f9e968489796f7e89efa8641511 WatchSource:0}: Error finding container 7d231e6adfc1c9652b86c547b9b0d4be1e622f9e968489796f7e89efa8641511: Status 404 returned error can't find the container with id 7d231e6adfc1c9652b86c547b9b0d4be1e622f9e968489796f7e89efa8641511
Sep 09 03:35:52.702479 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:52.702442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w" event={"ID":"fb6cb293-76b0-4f53-8d51-e38862d793e6","Type":"ContainerStarted","Data":"79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e"}
Sep 09 03:35:52.702954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:52.702488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w" event={"ID":"fb6cb293-76b0-4f53-8d51-e38862d793e6","Type":"ContainerStarted","Data":"7d231e6adfc1c9652b86c547b9b0d4be1e622f9e968489796f7e89efa8641511"}
Sep 09 03:35:52.723299 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:52.723256    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w" podStartSLOduration=1.723241164 podStartE2EDuration="1.723241164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:35:52.72213151 +0000 UTC m=+1363.421895473" watchObservedRunningTime="2026-09-09 03:35:52.723241164 +0000 UTC m=+1363.423005126"
Sep 09 03:35:53.710539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:53.710511    2580 generic.go:356] "Generic (PLEG): container finished" podID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerID="e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a" exitCode=0
Sep 09 03:35:53.710979 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:53.710589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" event={"ID":"c41d7cdc-57d2-427d-ba6d-4260c30a3379","Type":"ContainerDied","Data":"e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"}
Sep 09 03:35:53.711052 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:53.711036    2580 scope.go:117] "RemoveContainer" containerID="e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"
Sep 09 03:35:53.712045 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:53.712018    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:53.712111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:53.712098    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:53.712153 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:53.712133    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:35:54.719655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:54.719618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" event={"ID":"c41d7cdc-57d2-427d-ba6d-4260c30a3379","Type":"ContainerStarted","Data":"5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d"}
Sep 09 03:35:55.352226 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:55.352137    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:35:55.352454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:55.352429    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 09 03:35:55.352561 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:55.352486    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 09 03:35:56.352079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:56.352041    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 09 03:35:56.352589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:56.352095    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 09 03:35:57.352409 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:57.352371    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 09 03:35:57.352905 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:57.352439    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 09 03:35:58.351770 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:58.351731    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 09 03:35:58.352078 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:58.352046    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 09 03:35:59.351703 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:59.351642    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 09 03:35:59.352263 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:35:59.352229    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 09 03:36:00.356717 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:00.356687    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:36:00.357124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:00.357045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:36:00.357821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:00.357797    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:36:01.242995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.242963    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:01.245653 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.245637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:01.248312 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.248286    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k4rsk\""
Sep 09 03:36:01.254083 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.253794    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:01.361080 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.361054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc82w\" (UniqueName: \"kubernetes.io/projected/66e256ab-88d7-4e49-87c7-680bdf5a5821-kube-api-access-bc82w\") pod \"test-curl-7647-139627984442832\" (UID: \"66e256ab-88d7-4e49-87c7-680bdf5a5821\") " pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:01.461911 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.461881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc82w\" (UniqueName: \"kubernetes.io/projected/66e256ab-88d7-4e49-87c7-680bdf5a5821-kube-api-access-bc82w\") pod \"test-curl-7647-139627984442832\" (UID: \"66e256ab-88d7-4e49-87c7-680bdf5a5821\") " pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:01.471651 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.471618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc82w\" (UniqueName: \"kubernetes.io/projected/66e256ab-88d7-4e49-87c7-680bdf5a5821-kube-api-access-bc82w\") pod \"test-curl-7647-139627984442832\" (UID: \"66e256ab-88d7-4e49-87c7-680bdf5a5821\") " pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:01.557997 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.557937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:01.811020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.810946    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:36:01.811020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.811001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:36:01.823887 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.823857    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:36:01.831054 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.831037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:01.833929 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:36:01.833905    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66e256ab_88d7_4e49_87c7_680bdf5a5821.slice/crio-388db85d317d7cd252148e1d1e489c8b2770f67788914f6fa0299fb33550d1c7 WatchSource:0}: Error finding container 388db85d317d7cd252148e1d1e489c8b2770f67788914f6fa0299fb33550d1c7: Status 404 returned error can't find the container with id 388db85d317d7cd252148e1d1e489c8b2770f67788914f6fa0299fb33550d1c7
Sep 09 03:36:01.835597 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:01.835583    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:36:02.781510 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:02.781476    2580 generic.go:356] "Generic (PLEG): container finished" podID="66e256ab-88d7-4e49-87c7-680bdf5a5821" containerID="93a65fd338c1efe2cd17fe7ace2f2370dbb61c67a973c967cd5fd9349d6e3531" exitCode=0
Sep 09 03:36:02.781904 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:02.781564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139627984442832" event={"ID":"66e256ab-88d7-4e49-87c7-680bdf5a5821","Type":"ContainerDied","Data":"93a65fd338c1efe2cd17fe7ace2f2370dbb61c67a973c967cd5fd9349d6e3531"}
Sep 09 03:36:02.781904 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:02.781602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139627984442832" event={"ID":"66e256ab-88d7-4e49-87c7-680bdf5a5821","Type":"ContainerStarted","Data":"388db85d317d7cd252148e1d1e489c8b2770f67788914f6fa0299fb33550d1c7"}
Sep 09 03:36:02.793088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:02.793065    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:36:03.920057 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:03.920034    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:03.963550 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:03.963523    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7647-139627984442832_66e256ab-88d7-4e49-87c7-680bdf5a5821/test-curl-7647-139627984442832/0.log"
Sep 09 03:36:03.984799 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:03.984775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc82w\" (UniqueName: \"kubernetes.io/projected/66e256ab-88d7-4e49-87c7-680bdf5a5821-kube-api-access-bc82w\") pod \"66e256ab-88d7-4e49-87c7-680bdf5a5821\" (UID: \"66e256ab-88d7-4e49-87c7-680bdf5a5821\") "
Sep 09 03:36:03.987119 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:03.987095    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66e256ab-88d7-4e49-87c7-680bdf5a5821-kube-api-access-bc82w" (OuterVolumeSpecName: "kube-api-access-bc82w") pod "66e256ab-88d7-4e49-87c7-680bdf5a5821" (UID: "66e256ab-88d7-4e49-87c7-680bdf5a5821"). InnerVolumeSpecName "kube-api-access-bc82w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:36:03.990541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:03.990517    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:03.996994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:03.996975    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:04.086418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.086367    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc82w\" (UniqueName: \"kubernetes.io/projected/66e256ab-88d7-4e49-87c7-680bdf5a5821-kube-api-access-bc82w\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:36:04.114426 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.114406    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:04.114864 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.114850    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66e256ab-88d7-4e49-87c7-680bdf5a5821" containerName="test-curl-7647-139627984442832"
Sep 09 03:36:04.114925 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.114866    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66e256ab-88d7-4e49-87c7-680bdf5a5821" containerName="test-curl-7647-139627984442832"
Sep 09 03:36:04.114963 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.114936    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66e256ab-88d7-4e49-87c7-680bdf5a5821" containerName="test-curl-7647-139627984442832"
Sep 09 03:36:04.117949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.117935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:04.126574 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.126242    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:04.187336 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.187312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfksg\" (UniqueName: \"kubernetes.io/projected/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4-kube-api-access-tfksg\") pod \"test-curl-7647-139627984442832\" (UID: \"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4\") " pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:04.288610 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.288585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfksg\" (UniqueName: \"kubernetes.io/projected/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4-kube-api-access-tfksg\") pod \"test-curl-7647-139627984442832\" (UID: \"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4\") " pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:04.298485 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.298460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfksg\" (UniqueName: \"kubernetes.io/projected/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4-kube-api-access-tfksg\") pod \"test-curl-7647-139627984442832\" (UID: \"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4\") " pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:04.429541 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.429469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:04.709016 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.708991    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:04.710483 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:36:04.710456    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f9a3cd8_bd70_42b8_9d00_5807e79ac5b4.slice/crio-2d56d0939db785c0cc74f9159e45aea66d406ba672d4ce230e190ec791cf5763 WatchSource:0}: Error finding container 2d56d0939db785c0cc74f9159e45aea66d406ba672d4ce230e190ec791cf5763: Status 404 returned error can't find the container with id 2d56d0939db785c0cc74f9159e45aea66d406ba672d4ce230e190ec791cf5763
Sep 09 03:36:04.794557 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.794532    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:04.794712 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.794559    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="388db85d317d7cd252148e1d1e489c8b2770f67788914f6fa0299fb33550d1c7"
Sep 09 03:36:04.795815 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.795792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139627984442832" event={"ID":"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4","Type":"ContainerStarted","Data":"2d56d0939db785c0cc74f9159e45aea66d406ba672d4ce230e190ec791cf5763"}
Sep 09 03:36:04.806391 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:04.806365    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7647-139627984442832" oldPodUID="66e256ab-88d7-4e49-87c7-680bdf5a5821" podUID="6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4"
Sep 09 03:36:05.790001 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:05.789968    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66e256ab-88d7-4e49-87c7-680bdf5a5821" path="/var/lib/kubelet/pods/66e256ab-88d7-4e49-87c7-680bdf5a5821/volumes"
Sep 09 03:36:05.803345 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:05.803322    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4" containerID="9766fefd2e1f768e6c0844e31916bce8e13c3bb2e4c29878cf8aafaab26fd7ab" exitCode=0
Sep 09 03:36:05.803476 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:05.803415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139627984442832" event={"ID":"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4","Type":"ContainerDied","Data":"9766fefd2e1f768e6c0844e31916bce8e13c3bb2e4c29878cf8aafaab26fd7ab"}
Sep 09 03:36:06.957767 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:06.957744    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:06.981032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:06.981008    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7647-139627984442832_6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4/test-curl-7647-139627984442832/0.log"
Sep 09 03:36:07.005535 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.005509    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:07.008875 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.008852    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7647-139627984442832"]
Sep 09 03:36:07.017249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.017233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfksg\" (UniqueName: \"kubernetes.io/projected/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4-kube-api-access-tfksg\") pod \"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4\" (UID: \"6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4\") "
Sep 09 03:36:07.019273 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.019243    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4-kube-api-access-tfksg" (OuterVolumeSpecName: "kube-api-access-tfksg") pod "6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4" (UID: "6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4"). InnerVolumeSpecName "kube-api-access-tfksg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:36:07.118980 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.118921    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfksg\" (UniqueName: \"kubernetes.io/projected/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4-kube-api-access-tfksg\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:36:07.123787 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.123766    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7647-139627984443408"]
Sep 09 03:36:07.124190 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.124177    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4" containerName="test-curl-7647-139627984442832"
Sep 09 03:36:07.124236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.124192    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4" containerName="test-curl-7647-139627984442832"
Sep 09 03:36:07.124284 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.124273    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4" containerName="test-curl-7647-139627984442832"
Sep 09 03:36:07.127387 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.127368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:07.133913 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.133890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139627984443408"]
Sep 09 03:36:07.219910 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.219886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r82x7\" (UniqueName: \"kubernetes.io/projected/a847fafd-fe67-4cb6-ab9f-96cff38a5803-kube-api-access-r82x7\") pod \"test-curl-7647-139627984443408\" (UID: \"a847fafd-fe67-4cb6-ab9f-96cff38a5803\") " pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:07.321317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.321294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r82x7\" (UniqueName: \"kubernetes.io/projected/a847fafd-fe67-4cb6-ab9f-96cff38a5803-kube-api-access-r82x7\") pod \"test-curl-7647-139627984443408\" (UID: \"a847fafd-fe67-4cb6-ab9f-96cff38a5803\") " pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:07.331769 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.331734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r82x7\" (UniqueName: \"kubernetes.io/projected/a847fafd-fe67-4cb6-ab9f-96cff38a5803-kube-api-access-r82x7\") pod \"test-curl-7647-139627984443408\" (UID: \"a847fafd-fe67-4cb6-ab9f-96cff38a5803\") " pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:07.440833 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.440765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:07.713868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.713844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7647-139627984443408"]
Sep 09 03:36:07.714897 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:36:07.714874    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda847fafd_fe67_4cb6_ab9f_96cff38a5803.slice/crio-5c052285c597c2a48016723aa564ad85f9c404c8848ef5ee535614101d0c49c8 WatchSource:0}: Error finding container 5c052285c597c2a48016723aa564ad85f9c404c8848ef5ee535614101d0c49c8: Status 404 returned error can't find the container with id 5c052285c597c2a48016723aa564ad85f9c404c8848ef5ee535614101d0c49c8
Sep 09 03:36:07.790502 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.790471    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4" path="/var/lib/kubelet/pods/6f9a3cd8-bd70-42b8-9d00-5807e79ac5b4/volumes"
Sep 09 03:36:07.815840 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.815814    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139627984443408" event={"ID":"a847fafd-fe67-4cb6-ab9f-96cff38a5803","Type":"ContainerStarted","Data":"5c052285c597c2a48016723aa564ad85f9c404c8848ef5ee535614101d0c49c8"}
Sep 09 03:36:07.817024 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.817005    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984442832"
Sep 09 03:36:07.817127 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:07.817051    2580 scope.go:117] "RemoveContainer" containerID="9766fefd2e1f768e6c0844e31916bce8e13c3bb2e4c29878cf8aafaab26fd7ab"
Sep 09 03:36:08.824712 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:08.824666    2580 generic.go:356] "Generic (PLEG): container finished" podID="a847fafd-fe67-4cb6-ab9f-96cff38a5803" containerID="458d42de9a3e4e1b8a7ba53d35afefda2c5f90153d7fcd25abc8fe168d45c428" exitCode=0
Sep 09 03:36:08.825071 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:08.824775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7647-139627984443408" event={"ID":"a847fafd-fe67-4cb6-ab9f-96cff38a5803","Type":"ContainerDied","Data":"458d42de9a3e4e1b8a7ba53d35afefda2c5f90153d7fcd25abc8fe168d45c428"}
Sep 09 03:36:09.937257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:09.937226    2580 scope.go:117] "RemoveContainer" containerID="e40a4af1960a9fed40675a64f3419d10955859f18d66194f7faef3f88aac85a3"
Sep 09 03:36:09.972397 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:09.972375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:09.996811 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:09.996782    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7647-139627984443408_a847fafd-fe67-4cb6-ab9f-96cff38a5803/test-curl-7647-139627984443408/0.log"
Sep 09 03:36:10.022059 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.022032    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7647-139627984443408"]
Sep 09 03:36:10.026447 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.026425    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7647-139627984443408"]
Sep 09 03:36:10.045803 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.045784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r82x7\" (UniqueName: \"kubernetes.io/projected/a847fafd-fe67-4cb6-ab9f-96cff38a5803-kube-api-access-r82x7\") pod \"a847fafd-fe67-4cb6-ab9f-96cff38a5803\" (UID: \"a847fafd-fe67-4cb6-ab9f-96cff38a5803\") "
Sep 09 03:36:10.047937 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.047907    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a847fafd-fe67-4cb6-ab9f-96cff38a5803-kube-api-access-r82x7" (OuterVolumeSpecName: "kube-api-access-r82x7") pod "a847fafd-fe67-4cb6-ab9f-96cff38a5803" (UID: "a847fafd-fe67-4cb6-ab9f-96cff38a5803"). InnerVolumeSpecName "kube-api-access-r82x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:36:10.146695 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.146610    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r82x7\" (UniqueName: \"kubernetes.io/projected/a847fafd-fe67-4cb6-ab9f-96cff38a5803-kube-api-access-r82x7\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:36:10.841450 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.841418    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:10.841623 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:10.841407    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c052285c597c2a48016723aa564ad85f9c404c8848ef5ee535614101d0c49c8"
Sep 09 03:36:10.852723 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:36:10.852693    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7647-139627984443408\" is forbidden: User \"system:node:ip-10-0-142-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-178.ec2.internal' and this object" podUID="a847fafd-fe67-4cb6-ab9f-96cff38a5803" pod="opendatahub/test-curl-7647-139627984443408"
Sep 09 03:36:11.789416 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:11.789374    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a847fafd-fe67-4cb6-ab9f-96cff38a5803" path="/var/lib/kubelet/pods/a847fafd-fe67-4cb6-ab9f-96cff38a5803/volumes"
Sep 09 03:36:14.159740 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.159708    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"]
Sep 09 03:36:14.160132 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.160120    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a847fafd-fe67-4cb6-ab9f-96cff38a5803" containerName="test-curl-7647-139627984443408"
Sep 09 03:36:14.160180 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.160134    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a847fafd-fe67-4cb6-ab9f-96cff38a5803" containerName="test-curl-7647-139627984443408"
Sep 09 03:36:14.160227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.160218    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a847fafd-fe67-4cb6-ab9f-96cff38a5803" containerName="test-curl-7647-139627984443408"
Sep 09 03:36:14.163386 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.163364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.165966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.165949    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-123302cc-openshift-default-dockercfg-fx68n\""
Sep 09 03:36:14.178847 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.178823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"]
Sep 09 03:36:14.285571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-certs\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-envoy\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285822 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wttt\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-kube-api-access-9wttt\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285822 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-socket\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285822 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-data\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285822 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-podinfo\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-credential-socket\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-token\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.285990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.285948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istiod-ca-cert\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.386769 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.386733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-certs\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.386931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.386779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-envoy\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.386931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.386820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wttt\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-kube-api-access-9wttt\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.386931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.386871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-socket\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.386931 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.386891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-data\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387132 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-podinfo\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387193 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-credential-socket\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387248 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-certs\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387248 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-token\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-socket\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387407 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istiod-ca-cert\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387526 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-credential-socket\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387526 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-data\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.387943 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.387870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istiod-ca-cert\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.389577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.389552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-envoy\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.389755 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.389741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-podinfo\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.395465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.395445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wttt\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-kube-api-access-9wttt\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.396047 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.396031    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-token\") pod \"e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") " pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.474293 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.474267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:14.770517 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.770486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"]
Sep 09 03:36:14.770827 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:36:14.770807    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58d1a2a_bd79_4d10_b2eb_61fe8624fe88.slice/crio-2d699c9f5eea53c16161bfa75843c4eea03a9db3d613eb773a565043b308e66c WatchSource:0}: Error finding container 2d699c9f5eea53c16161bfa75843c4eea03a9db3d613eb773a565043b308e66c: Status 404 returned error can't find the container with id 2d699c9f5eea53c16161bfa75843c4eea03a9db3d613eb773a565043b308e66c
Sep 09 03:36:14.773497 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.773462    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:36:14.773583 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.773528    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:36:14.773583 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.773555    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:36:14.867453 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.867426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" event={"ID":"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88","Type":"ContainerStarted","Data":"991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"}
Sep 09 03:36:14.867595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.867471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" event={"ID":"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88","Type":"ContainerStarted","Data":"2d699c9f5eea53c16161bfa75843c4eea03a9db3d613eb773a565043b308e66c"}
Sep 09 03:36:14.903816 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:14.903767    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podStartSLOduration=0.903751703 podStartE2EDuration="903.751703ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:36:14.899844521 +0000 UTC m=+1385.599608482" watchObservedRunningTime="2026-09-09 03:36:14.903751703 +0000 UTC m=+1385.603515665"
Sep 09 03:36:15.475237 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:15.475209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:15.480958 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:15.480936    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:15.874013 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:15.873934    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:15.874884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:15.874858    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:39.041738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:39.041709    2580 generic.go:356] "Generic (PLEG): container finished" podID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerID="991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c" exitCode=0
Sep 09 03:36:39.042155 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:39.041784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" event={"ID":"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88","Type":"ContainerDied","Data":"991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"}
Sep 09 03:36:39.042333 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:39.042315    2580 scope.go:117] "RemoveContainer" containerID="991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"
Sep 09 03:36:39.043513 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:39.043485    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:36:39.043577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:39.043556    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:36:39.043615 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:39.043589    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:36:40.049945 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:40.049914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" event={"ID":"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88","Type":"ContainerStarted","Data":"ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781"}
Sep 09 03:36:40.475174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:40.475143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:40.475330 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:40.475294    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 09 03:36:40.475398 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:40.475347    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 09 03:36:41.475365 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:41.475329    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 09 03:36:41.475904 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:41.475396    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 09 03:36:42.474877 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:42.474842    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 09 03:36:42.475051 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:42.474898    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 09 03:36:43.475570 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:43.475528    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 09 03:36:43.476052 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:43.475594    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 09 03:36:44.475125 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:44.475089    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 09 03:36:44.475299 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:44.475144    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 09 03:36:45.475481 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:45.475448    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 09 03:36:45.475885 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:45.475497    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 09 03:36:45.874456 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:45.874367    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:46.479139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:46.479104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:36:46.479975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:36:46.479956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:38:13.291943 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:13.291911    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"]
Sep 09 03:38:14.300708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:14.300661    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"]
Sep 09 03:38:28.312314 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.312271    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk" podUID="83ecedd9-008e-4375-bda9-d1766b7e9b4e" containerName="main" containerID="cri-o://ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd" gracePeriod=45
Sep 09 03:38:28.577142 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.577086    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:38:28.618868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.618847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-home\") pod \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") "
Sep 09 03:38:28.619020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.618912    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-model-cache\") pod \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") "
Sep 09 03:38:28.619020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.618975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tls-certs\") pod \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") "
Sep 09 03:38:28.619020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.618997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2snr\" (UniqueName: \"kubernetes.io/projected/83ecedd9-008e-4375-bda9-d1766b7e9b4e-kube-api-access-j2snr\") pod \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") "
Sep 09 03:38:28.619020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-dshm\") pod \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") "
Sep 09 03:38:28.619259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619042    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tmp-dir\") pod \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\" (UID: \"83ecedd9-008e-4375-bda9-d1766b7e9b4e\") "
Sep 09 03:38:28.619259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619100    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-home" (OuterVolumeSpecName: "home") pod "83ecedd9-008e-4375-bda9-d1766b7e9b4e" (UID: "83ecedd9-008e-4375-bda9-d1766b7e9b4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:28.619259 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "83ecedd9-008e-4375-bda9-d1766b7e9b4e" (UID: "83ecedd9-008e-4375-bda9-d1766b7e9b4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:28.619460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83ecedd9-008e-4375-bda9-d1766b7e9b4e" (UID: "83ecedd9-008e-4375-bda9-d1766b7e9b4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:28.619525 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619500    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:28.619525 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.619521    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:28.621398 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.621364    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83ecedd9-008e-4375-bda9-d1766b7e9b4e-kube-api-access-j2snr" (OuterVolumeSpecName: "kube-api-access-j2snr") pod "83ecedd9-008e-4375-bda9-d1766b7e9b4e" (UID: "83ecedd9-008e-4375-bda9-d1766b7e9b4e"). InnerVolumeSpecName "kube-api-access-j2snr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:38:28.621716 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.621645    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-dshm" (OuterVolumeSpecName: "dshm") pod "83ecedd9-008e-4375-bda9-d1766b7e9b4e" (UID: "83ecedd9-008e-4375-bda9-d1766b7e9b4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:28.621831 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.621805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83ecedd9-008e-4375-bda9-d1766b7e9b4e" (UID: "83ecedd9-008e-4375-bda9-d1766b7e9b4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:38:28.720661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.720639    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:28.720661 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.720661    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2snr\" (UniqueName: \"kubernetes.io/projected/83ecedd9-008e-4375-bda9-d1766b7e9b4e-kube-api-access-j2snr\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:28.720798 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.720692    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:28.720798 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.720701    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ecedd9-008e-4375-bda9-d1766b7e9b4e-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:28.754535 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.754506    2580 generic.go:356] "Generic (PLEG): container finished" podID="83ecedd9-008e-4375-bda9-d1766b7e9b4e" containerID="ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd" exitCode=0
Sep 09 03:38:28.754643 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.754549    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"
Sep 09 03:38:28.754643 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.754590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk" event={"ID":"83ecedd9-008e-4375-bda9-d1766b7e9b4e","Type":"ContainerDied","Data":"ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd"}
Sep 09 03:38:28.754643 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.754631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk" event={"ID":"83ecedd9-008e-4375-bda9-d1766b7e9b4e","Type":"ContainerDied","Data":"89403e40884bcf904e35fdf90c5758e626ba7988262dac927f41946b835ab0cf"}
Sep 09 03:38:28.754808 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.754649    2580 scope.go:117] "RemoveContainer" containerID="ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd"
Sep 09 03:38:28.765296 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.765280    2580 scope.go:117] "RemoveContainer" containerID="ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd"
Sep 09 03:38:28.765566 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:38:28.765545    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd\": container with ID starting with ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd not found: ID does not exist" containerID="ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd"
Sep 09 03:38:28.765696 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.765574    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd"} err="failed to get container status \"ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd\": rpc error: code = NotFound desc = could not find container \"ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd\": container with ID starting with ad212b6a069706108d492097d73a133b637f151269cba1e71679a029116803dd not found: ID does not exist"
Sep 09 03:38:28.779010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.778985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"]
Sep 09 03:38:28.789311 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:28.787787    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/auth-test-model-5d8e85-kserve-7785689fdc-rjjhk"]
Sep 09 03:38:29.320582 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.320538    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w" podUID="fb6cb293-76b0-4f53-8d51-e38862d793e6" containerName="main" containerID="cri-o://79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e" gracePeriod=45
Sep 09 03:38:29.578856 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.578793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:38:29.629395 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629363    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-model-cache\") pod \"fb6cb293-76b0-4f53-8d51-e38862d793e6\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") "
Sep 09 03:38:29.629555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629405    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6cb293-76b0-4f53-8d51-e38862d793e6-tls-certs\") pod \"fb6cb293-76b0-4f53-8d51-e38862d793e6\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") "
Sep 09 03:38:29.629555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629448    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-tmp-dir\") pod \"fb6cb293-76b0-4f53-8d51-e38862d793e6\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") "
Sep 09 03:38:29.629555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629496    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwcck\" (UniqueName: \"kubernetes.io/projected/fb6cb293-76b0-4f53-8d51-e38862d793e6-kube-api-access-hwcck\") pod \"fb6cb293-76b0-4f53-8d51-e38862d793e6\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") "
Sep 09 03:38:29.629734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629579    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-dshm\") pod \"fb6cb293-76b0-4f53-8d51-e38862d793e6\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") "
Sep 09 03:38:29.629734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-home\") pod \"fb6cb293-76b0-4f53-8d51-e38862d793e6\" (UID: \"fb6cb293-76b0-4f53-8d51-e38862d793e6\") "
Sep 09 03:38:29.629734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629658    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb6cb293-76b0-4f53-8d51-e38862d793e6" (UID: "fb6cb293-76b0-4f53-8d51-e38862d793e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:29.629734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629703    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb6cb293-76b0-4f53-8d51-e38862d793e6" (UID: "fb6cb293-76b0-4f53-8d51-e38862d793e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:29.629956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629935    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:29.629956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629939    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-home" (OuterVolumeSpecName: "home") pod "fb6cb293-76b0-4f53-8d51-e38862d793e6" (UID: "fb6cb293-76b0-4f53-8d51-e38862d793e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:29.629956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.629950    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:29.631899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.631876    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6cb293-76b0-4f53-8d51-e38862d793e6-kube-api-access-hwcck" (OuterVolumeSpecName: "kube-api-access-hwcck") pod "fb6cb293-76b0-4f53-8d51-e38862d793e6" (UID: "fb6cb293-76b0-4f53-8d51-e38862d793e6"). InnerVolumeSpecName "kube-api-access-hwcck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:38:29.632018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.631988    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-dshm" (OuterVolumeSpecName: "dshm") pod "fb6cb293-76b0-4f53-8d51-e38862d793e6" (UID: "fb6cb293-76b0-4f53-8d51-e38862d793e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:29.632018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.631994    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb6cb293-76b0-4f53-8d51-e38862d793e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb6cb293-76b0-4f53-8d51-e38862d793e6" (UID: "fb6cb293-76b0-4f53-8d51-e38862d793e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:38:29.731043 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.731013    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwcck\" (UniqueName: \"kubernetes.io/projected/fb6cb293-76b0-4f53-8d51-e38862d793e6-kube-api-access-hwcck\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:29.731043 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.731037    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:29.731181 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.731052    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb6cb293-76b0-4f53-8d51-e38862d793e6-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:29.731181 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.731064    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6cb293-76b0-4f53-8d51-e38862d793e6-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:29.763132 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.763105    2580 generic.go:356] "Generic (PLEG): container finished" podID="fb6cb293-76b0-4f53-8d51-e38862d793e6" containerID="79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e" exitCode=0
Sep 09 03:38:29.763223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.763157    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"
Sep 09 03:38:29.763223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.763163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w" event={"ID":"fb6cb293-76b0-4f53-8d51-e38862d793e6","Type":"ContainerDied","Data":"79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e"}
Sep 09 03:38:29.763223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.763198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w" event={"ID":"fb6cb293-76b0-4f53-8d51-e38862d793e6","Type":"ContainerDied","Data":"7d231e6adfc1c9652b86c547b9b0d4be1e622f9e968489796f7e89efa8641511"}
Sep 09 03:38:29.763223 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.763218    2580 scope.go:117] "RemoveContainer" containerID="79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e"
Sep 09 03:38:29.775106 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.775083    2580 scope.go:117] "RemoveContainer" containerID="79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e"
Sep 09 03:38:29.775611 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:38:29.775589    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e\": container with ID starting with 79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e not found: ID does not exist" containerID="79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e"
Sep 09 03:38:29.775704 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.775619    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e"} err="failed to get container status \"79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e\": rpc error: code = NotFound desc = could not find container \"79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e\": container with ID starting with 79bd371e568c599ca2eb6d17389399279ffe1d8848becc5c4cd5a0571ff0fa5e not found: ID does not exist"
Sep 09 03:38:29.791705 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.791663    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83ecedd9-008e-4375-bda9-d1766b7e9b4e" path="/var/lib/kubelet/pods/83ecedd9-008e-4375-bda9-d1766b7e9b4e/volumes"
Sep 09 03:38:29.791954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.791941    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"]
Sep 09 03:38:29.791996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:29.791966    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/auth-test-model-c34148-kserve-744c6dcf87-4sh2w"]
Sep 09 03:38:31.788856 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:31.788785    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb6cb293-76b0-4f53-8d51-e38862d793e6" path="/var/lib/kubelet/pods/fb6cb293-76b0-4f53-8d51-e38862d793e6/volumes"
Sep 09 03:38:35.400870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.400836    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"]
Sep 09 03:38:35.401550 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.401533    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb6cb293-76b0-4f53-8d51-e38862d793e6" containerName="main"
Sep 09 03:38:35.401594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.401554    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6cb293-76b0-4f53-8d51-e38862d793e6" containerName="main"
Sep 09 03:38:35.401594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.401583    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ecedd9-008e-4375-bda9-d1766b7e9b4e" containerName="main"
Sep 09 03:38:35.401594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.401591    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ecedd9-008e-4375-bda9-d1766b7e9b4e" containerName="main"
Sep 09 03:38:35.401737 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.401725    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb6cb293-76b0-4f53-8d51-e38862d793e6" containerName="main"
Sep 09 03:38:35.401776 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.401742    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ecedd9-008e-4375-bda9-d1766b7e9b4e" containerName="main"
Sep 09 03:38:35.406666 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.406645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.410959 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.410937    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"kube-root-ca.crt\""
Sep 09 03:38:35.412982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.412961    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"default-dockercfg-r99j8\""
Sep 09 03:38:35.413321 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.412971    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"openshift-service-ca.crt\""
Sep 09 03:38:35.413627 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.412980    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d8e85\"/\"sub-test-model-5d8e85-kserve-self-signed-certs\""
Sep 09 03:38:35.420992 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.420459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"]
Sep 09 03:38:35.482163 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.482134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-tmp-dir\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.482283 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.482196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d949391-ec9d-4153-8e91-c07ae408528b-tls-certs\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.482326 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.482274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-model-cache\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.482326 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.482299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrc7g\" (UniqueName: \"kubernetes.io/projected/0d949391-ec9d-4153-8e91-c07ae408528b-kube-api-access-lrc7g\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.482326 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.482314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-dshm\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.482423 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.482361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-home\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583119 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-model-cache\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583119 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrc7g\" (UniqueName: \"kubernetes.io/projected/0d949391-ec9d-4153-8e91-c07ae408528b-kube-api-access-lrc7g\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-dshm\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-home\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-tmp-dir\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d949391-ec9d-4153-8e91-c07ae408528b-tls-certs\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-model-cache\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-home\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.583664 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.583649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-tmp-dir\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.585753 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.585731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-dshm\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.585904 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.585887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d949391-ec9d-4153-8e91-c07ae408528b-tls-certs\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.592018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.591993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrc7g\" (UniqueName: \"kubernetes.io/projected/0d949391-ec9d-4153-8e91-c07ae408528b-kube-api-access-lrc7g\") pod \"sub-test-model-5d8e85-kserve-697475c48f-nc2qz\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") " pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:35.722077 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:35.722048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:36.003008 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:36.002983    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"]
Sep 09 03:38:36.004362 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:38:36.004334    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d949391_ec9d_4153_8e91_c07ae408528b.slice/crio-2ec2a0b441d1353ba5483fe76dafa72ce7cd3bf069a3e4564580cce3996e169d WatchSource:0}: Error finding container 2ec2a0b441d1353ba5483fe76dafa72ce7cd3bf069a3e4564580cce3996e169d: Status 404 returned error can't find the container with id 2ec2a0b441d1353ba5483fe76dafa72ce7cd3bf069a3e4564580cce3996e169d
Sep 09 03:38:36.812374 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:36.812338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz" event={"ID":"0d949391-ec9d-4153-8e91-c07ae408528b","Type":"ContainerStarted","Data":"b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f"}
Sep 09 03:38:36.812762 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:36.812381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz" event={"ID":"0d949391-ec9d-4153-8e91-c07ae408528b","Type":"ContainerStarted","Data":"2ec2a0b441d1353ba5483fe76dafa72ce7cd3bf069a3e4564580cce3996e169d"}
Sep 09 03:38:36.830477 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:36.830422    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz" podStartSLOduration=1.830408234 podStartE2EDuration="1.830408234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:38:36.828938138 +0000 UTC m=+1527.528702101" watchObservedRunningTime="2026-09-09 03:38:36.830408234 +0000 UTC m=+1527.530172195"
Sep 09 03:38:41.927280 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:41.927245    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"]
Sep 09 03:38:41.927797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:41.927498    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" containerID="cri-o://ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781" gracePeriod=30
Sep 09 03:38:45.722875 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:45.722844    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:45.723253 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:45.722889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:45.739139 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:45.739116    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:45.878655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:45.878624    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:38:46.481612 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:46.481576    2580 patch_prober.go:28] interesting pod/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:38:46.481805 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:46.481640    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:38:47.183052 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.183030    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:38:47.287830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.287799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-token\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.287983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.287839    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-envoy\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.287983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.287877    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-data\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.287983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.287916    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-socket\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.287983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.287944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-podinfo\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.288189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.287986    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wttt\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-kube-api-access-9wttt\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.288189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288015    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-certs\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.288189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288037    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istiod-ca-cert\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.288189 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-credential-socket\") pod \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\" (UID: \"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88\") "
Sep 09 03:38:47.288467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288191    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-data" (OuterVolumeSpecName: "istio-data") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:47.288467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288207    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:47.288467 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288325    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:47.288637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288564    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.288637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288583    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.288637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288598    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.288847 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288730    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:38:47.289045 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.288985    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:47.290231 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.290207    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-token" (OuterVolumeSpecName: "istio-token") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:38:47.290429 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.290409    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-kube-api-access-9wttt" (OuterVolumeSpecName: "kube-api-access-9wttt") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "kube-api-access-9wttt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:38:47.290829 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.290809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:38:47.290829 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.290809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" (UID: "f58d1a2a-bd79-4d10-b2eb-61fe8624fe88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:38:47.389289 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.389263    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.389289 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.389285    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.389289 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.389294    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.389452 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.389311    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wttt\" (UniqueName: \"kubernetes.io/projected/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-kube-api-access-9wttt\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.389452 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.389320    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.389452 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.389329    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:38:47.881804 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.881770    2580 generic.go:356] "Generic (PLEG): container finished" podID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerID="ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781" exitCode=0
Sep 09 03:38:47.881953 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.881821    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"
Sep 09 03:38:47.881953 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.881850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" event={"ID":"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88","Type":"ContainerDied","Data":"ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781"}
Sep 09 03:38:47.881953 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.881900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4" event={"ID":"f58d1a2a-bd79-4d10-b2eb-61fe8624fe88","Type":"ContainerDied","Data":"2d699c9f5eea53c16161bfa75843c4eea03a9db3d613eb773a565043b308e66c"}
Sep 09 03:38:47.881953 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.881922    2580 scope.go:117] "RemoveContainer" containerID="ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781"
Sep 09 03:38:47.891734 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.891712    2580 scope.go:117] "RemoveContainer" containerID="991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"
Sep 09 03:38:47.900948 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.900933    2580 scope.go:117] "RemoveContainer" containerID="ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781"
Sep 09 03:38:47.901246 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:38:47.901228    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781\": container with ID starting with ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781 not found: ID does not exist" containerID="ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781"
Sep 09 03:38:47.901327 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.901257    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781"} err="failed to get container status \"ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781\": rpc error: code = NotFound desc = could not find container \"ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781\": container with ID starting with ce5d805ddc488596f55894c9c9246e647b22d933f9c3d74cfc1612378ef68781 not found: ID does not exist"
Sep 09 03:38:47.901327 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.901281    2580 scope.go:117] "RemoveContainer" containerID="991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"
Sep 09 03:38:47.901588 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:38:47.901569    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c\": container with ID starting with 991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c not found: ID does not exist" containerID="991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"
Sep 09 03:38:47.901633 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.901594    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c"} err="failed to get container status \"991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c\": rpc error: code = NotFound desc = could not find container \"991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c\": container with ID starting with 991a147b28c02288bdd36d5d1250e0e0156b1fc29e94b720b22c3d6850047c7c not found: ID does not exist"
Sep 09 03:38:47.902503 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.902482    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"]
Sep 09 03:38:47.913804 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:47.913784    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-123302cc-openshift-default-6c4d47dd6d-clsx4"]
Sep 09 03:38:49.790054 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:38:49.790023    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" path="/var/lib/kubelet/pods/f58d1a2a-bd79-4d10-b2eb-61fe8624fe88/volumes"
Sep 09 03:39:11.447901 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.447870    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"]
Sep 09 03:39:11.448317 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.448304    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy"
Sep 09 03:39:11.448361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.448318    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy"
Sep 09 03:39:11.448361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.448335    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy"
Sep 09 03:39:11.448361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.448341    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy"
Sep 09 03:39:11.448461 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.448400    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy"
Sep 09 03:39:11.448461 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.448407    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58d1a2a-bd79-4d10-b2eb-61fe8624fe88" containerName="istio-proxy"
Sep 09 03:39:11.452816 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.452796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.455357 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.455335    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"openshift-service-ca.crt\""
Sep 09 03:39:11.456149 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.456126    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"sub-test-model-c34148-kserve-self-signed-certs\""
Sep 09 03:39:11.456299 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.456154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"default-dockercfg-gwznt\""
Sep 09 03:39:11.456427 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.456237    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"kube-root-ca.crt\""
Sep 09 03:39:11.464146 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.464124    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"]
Sep 09 03:39:11.499880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.499857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbxb6\" (UniqueName: \"kubernetes.io/projected/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-kube-api-access-mbxb6\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.499991 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.499884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-home\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.499991 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.499926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-model-cache\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.500088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.499996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tmp-dir\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.500088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.500065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-dshm\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.500157 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.500088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tls-certs\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.601285 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.601253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tmp-dir\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.602163 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.602140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-dshm\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.602738 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.602720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tls-certs\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.603318 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.602013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tmp-dir\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.603489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.603353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbxb6\" (UniqueName: \"kubernetes.io/projected/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-kube-api-access-mbxb6\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.603648 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.603627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-home\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.603884 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.603866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-model-cache\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.604400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.603973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-home\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.604526 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.604296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-model-cache\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.606914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.606890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-dshm\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.607295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.607280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tls-certs\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.612152 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.612135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbxb6\" (UniqueName: \"kubernetes.io/projected/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-kube-api-access-mbxb6\") pod \"sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") " pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:11.763545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:11.763518    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:12.040328 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:12.040300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"]
Sep 09 03:39:12.041008 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:39:12.040969    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad3ab6c3_acbd_4878_a4c7_ef513ad1f975.slice/crio-2d91ee698cacbc20f85c327c5079e209b8d713e314fd87fecc229faf7895ea04 WatchSource:0}: Error finding container 2d91ee698cacbc20f85c327c5079e209b8d713e314fd87fecc229faf7895ea04: Status 404 returned error can't find the container with id 2d91ee698cacbc20f85c327c5079e209b8d713e314fd87fecc229faf7895ea04
Sep 09 03:39:13.037830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:13.037793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb" event={"ID":"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975","Type":"ContainerStarted","Data":"d196c1dd99bb0bb16dde59de38bd917d95edb77cd36b1b38c15ce56f95da26bb"}
Sep 09 03:39:13.038284 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:13.037842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb" event={"ID":"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975","Type":"ContainerStarted","Data":"2d91ee698cacbc20f85c327c5079e209b8d713e314fd87fecc229faf7895ea04"}
Sep 09 03:39:13.059344 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:13.059296    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb" podStartSLOduration=2.059282034 podStartE2EDuration="2.059282034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:39:13.057326832 +0000 UTC m=+1563.757090797" watchObservedRunningTime="2026-09-09 03:39:13.059282034 +0000 UTC m=+1563.759045995"
Sep 09 03:39:21.764316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:21.764285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:21.764731 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:21.764368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:21.776523 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:21.776504    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:22.102869 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:22.102791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:39:37.547996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.547952    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"]
Sep 09 03:39:37.554077 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.554051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.556952 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.556930    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7f0891-openshift-default-dockercfg-99sjm\""
Sep 09 03:39:37.564641 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.564618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"]
Sep 09 03:39:37.731658 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-certs\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.731861 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knttd\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-kube-api-access-knttd\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.731861 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-token\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.731861 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-envoy\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.732030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7acda3a1-8730-4c53-8583-cb10f69cf41f-istiod-ca-cert\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.732030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-credential-socket\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.732030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.731964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-socket\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.732194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.732022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-podinfo\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.732194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.732090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-data\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-podinfo\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-data\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-certs\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knttd\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-kube-api-access-knttd\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-token\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-envoy\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7acda3a1-8730-4c53-8583-cb10f69cf41f-istiod-ca-cert\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.833807 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-credential-socket\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.834110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.833826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-socket\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.834110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.834062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-certs\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.834249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.834223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-socket\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.834320 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.834256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-data\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.834498 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.834467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-credential-socket\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.834746 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.834725    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7acda3a1-8730-4c53-8583-cb10f69cf41f-istiod-ca-cert\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.836197 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.836179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-envoy\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.836289 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.836213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-podinfo\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.851183 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.851160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-token\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.851373 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.851355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knttd\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-kube-api-access-knttd\") pod \"e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") " pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:37.867466 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:37.867439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:38.149782 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:38.149755    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"]
Sep 09 03:39:38.150225 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:39:38.150203    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7acda3a1_8730_4c53_8583_cb10f69cf41f.slice/crio-c8294230bb8ccb765f493d3a8241e943bb944f2b25557eba3d511796f21bed26 WatchSource:0}: Error finding container c8294230bb8ccb765f493d3a8241e943bb944f2b25557eba3d511796f21bed26: Status 404 returned error can't find the container with id c8294230bb8ccb765f493d3a8241e943bb944f2b25557eba3d511796f21bed26
Sep 09 03:39:38.152280 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:38.152245    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:39:38.152363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:38.152329    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:39:38.152404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:38.152367    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:39:38.190831 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:38.190802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" event={"ID":"7acda3a1-8730-4c53-8583-cb10f69cf41f","Type":"ContainerStarted","Data":"c8294230bb8ccb765f493d3a8241e943bb944f2b25557eba3d511796f21bed26"}
Sep 09 03:39:39.199080 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:39.199048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" event={"ID":"7acda3a1-8730-4c53-8583-cb10f69cf41f","Type":"ContainerStarted","Data":"4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"}
Sep 09 03:39:39.225990 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:39.225942    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podStartSLOduration=2.225926562 podStartE2EDuration="2.225926562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:39:39.223922438 +0000 UTC m=+1589.923686403" watchObservedRunningTime="2026-09-09 03:39:39.225926562 +0000 UTC m=+1589.925690523"
Sep 09 03:39:39.868111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:39.868082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:39.873708 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:39.873684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:40.205741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:40.205642    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:39:40.206774 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:39:40.206754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:40:03.352418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:03.352385    2580 generic.go:356] "Generic (PLEG): container finished" podID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerID="4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642" exitCode=0
Sep 09 03:40:03.352823 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:03.352434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" event={"ID":"7acda3a1-8730-4c53-8583-cb10f69cf41f","Type":"ContainerDied","Data":"4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"}
Sep 09 03:40:03.352876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:03.352838    2580 scope.go:117] "RemoveContainer" containerID="4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"
Sep 09 03:40:03.354021 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:03.353983    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:03.354112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:03.354061    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:03.354112 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:03.354092    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:04.361182 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:04.361149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" event={"ID":"7acda3a1-8730-4c53-8583-cb10f69cf41f","Type":"ContainerStarted","Data":"ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1"}
Sep 09 03:40:04.695240 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:04.695167    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"]
Sep 09 03:40:04.867856 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:04.867823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:40:04.868087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:04.868062    2580 patch_prober.go:28] interesting pod/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 09 03:40:04.868175 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:04.868121    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 09 03:40:05.867993 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:05.867943    2580 patch_prober.go:28] interesting pod/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 09 03:40:05.868399 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:05.868006    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 09 03:40:06.200543 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:06.200466    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"]
Sep 09 03:40:06.868124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:06.868091    2580 patch_prober.go:28] interesting pod/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 09 03:40:06.868510 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:06.868145    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 09 03:40:07.067637 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.067605    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"]
Sep 09 03:40:07.070227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.070209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.076096 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.076074    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1c4cb1-openshift-default-dockercfg-w2fnv\""
Sep 09 03:40:07.090547 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.090527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"]
Sep 09 03:40:07.222359 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-socket\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istiod-ca-cert\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-data\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-certs\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-token\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222539 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-envoy\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222746 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-podinfo\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222746 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-credential-socket\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.222746 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.222645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsn59\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-kube-api-access-nsn59\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.323752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.323722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-socket\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.323752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.323755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istiod-ca-cert\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.323966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.323940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-data\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324007 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.323983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-certs\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324043 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-token\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324078 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-envoy\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324134 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-podinfo\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324186 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-socket\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-credential-socket\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324245 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsn59\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-kube-api-access-nsn59\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324349 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-data\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324349 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-certs\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-credential-socket\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.324530 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.324510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istiod-ca-cert\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.330201 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.330167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-podinfo\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.335692 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.333739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsn59\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-kube-api-access-nsn59\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.335692 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.334202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-token\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.335949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.335923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-envoy\") pod \"e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") " pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.381173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.381152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:07.687655 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.687626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"]
Sep 09 03:40:07.688360 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:40:07.688332    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd070c2_d40a_44f8_a5b3_7c6b88867385.slice/crio-306429e4016a29a53d19db721231fa56c3f15607d250bcadbee0707c37e8b072 WatchSource:0}: Error finding container 306429e4016a29a53d19db721231fa56c3f15607d250bcadbee0707c37e8b072: Status 404 returned error can't find the container with id 306429e4016a29a53d19db721231fa56c3f15607d250bcadbee0707c37e8b072
Sep 09 03:40:07.690795 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.690720    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:07.690983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.690949    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:07.691054 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.691006    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:07.868921 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.868880    2580 patch_prober.go:28] interesting pod/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 09 03:40:07.869313 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:07.868947    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 09 03:40:08.388587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:08.388543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" event={"ID":"dbd070c2-d40a-44f8-a5b3-7c6b88867385","Type":"ContainerStarted","Data":"ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"}
Sep 09 03:40:08.388801 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:08.388643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" event={"ID":"dbd070c2-d40a-44f8-a5b3-7c6b88867385","Type":"ContainerStarted","Data":"306429e4016a29a53d19db721231fa56c3f15607d250bcadbee0707c37e8b072"}
Sep 09 03:40:08.412795 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:08.412745    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podStartSLOduration=1.412730961 podStartE2EDuration="1.412730961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:40:08.409521844 +0000 UTC m=+1619.109285805" watchObservedRunningTime="2026-09-09 03:40:08.412730961 +0000 UTC m=+1619.112494924"
Sep 09 03:40:08.868940 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:08.868904    2580 patch_prober.go:28] interesting pod/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 09 03:40:08.869320 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:08.868987    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 09 03:40:09.382252 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:09.382218    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:09.387752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:09.387727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:09.394607 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:09.394586    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:09.395215 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:09.395195    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:09.868744 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:09.868712    2580 patch_prober.go:28] interesting pod/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 09 03:40:09.868911 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:09.868761    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 09 03:40:10.206019 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:10.205933    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:40:10.872169 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:10.872142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:40:10.873369 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:10.873352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:40:19.715361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:19.715298    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz" podUID="0d949391-ec9d-4153-8e91-c07ae408528b" containerName="main" containerID="cri-o://b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f" gracePeriod=45
Sep 09 03:40:19.991962 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:19.991937    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:40:20.062707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062663    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-dshm\") pod \"0d949391-ec9d-4153-8e91-c07ae408528b\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") "
Sep 09 03:40:20.062874 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-model-cache\") pod \"0d949391-ec9d-4153-8e91-c07ae408528b\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") "
Sep 09 03:40:20.062874 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062783    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrc7g\" (UniqueName: \"kubernetes.io/projected/0d949391-ec9d-4153-8e91-c07ae408528b-kube-api-access-lrc7g\") pod \"0d949391-ec9d-4153-8e91-c07ae408528b\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") "
Sep 09 03:40:20.062874 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-home\") pod \"0d949391-ec9d-4153-8e91-c07ae408528b\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") "
Sep 09 03:40:20.063039 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062876    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d949391-ec9d-4153-8e91-c07ae408528b-tls-certs\") pod \"0d949391-ec9d-4153-8e91-c07ae408528b\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") "
Sep 09 03:40:20.063039 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-tmp-dir\") pod \"0d949391-ec9d-4153-8e91-c07ae408528b\" (UID: \"0d949391-ec9d-4153-8e91-c07ae408528b\") "
Sep 09 03:40:20.063039 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.062958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d949391-ec9d-4153-8e91-c07ae408528b" (UID: "0d949391-ec9d-4153-8e91-c07ae408528b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:20.063172 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.063085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-home" (OuterVolumeSpecName: "home") pod "0d949391-ec9d-4153-8e91-c07ae408528b" (UID: "0d949391-ec9d-4153-8e91-c07ae408528b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:20.063268 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.063238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d949391-ec9d-4153-8e91-c07ae408528b" (UID: "0d949391-ec9d-4153-8e91-c07ae408528b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:20.063392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.063374    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:20.063454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.063394    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:20.063454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.063423    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:20.064968 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.064943    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-dshm" (OuterVolumeSpecName: "dshm") pod "0d949391-ec9d-4153-8e91-c07ae408528b" (UID: "0d949391-ec9d-4153-8e91-c07ae408528b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:20.065075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.065056    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d949391-ec9d-4153-8e91-c07ae408528b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d949391-ec9d-4153-8e91-c07ae408528b" (UID: "0d949391-ec9d-4153-8e91-c07ae408528b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:40:20.065118 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.065081    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d949391-ec9d-4153-8e91-c07ae408528b-kube-api-access-lrc7g" (OuterVolumeSpecName: "kube-api-access-lrc7g") pod "0d949391-ec9d-4153-8e91-c07ae408528b" (UID: "0d949391-ec9d-4153-8e91-c07ae408528b"). InnerVolumeSpecName "kube-api-access-lrc7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:40:20.164663 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.164593    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrc7g\" (UniqueName: \"kubernetes.io/projected/0d949391-ec9d-4153-8e91-c07ae408528b-kube-api-access-lrc7g\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:20.164663 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.164627    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d949391-ec9d-4153-8e91-c07ae408528b-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:20.164663 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.164640    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d949391-ec9d-4153-8e91-c07ae408528b-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:20.470988 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.470960    2580 generic.go:356] "Generic (PLEG): container finished" podID="0d949391-ec9d-4153-8e91-c07ae408528b" containerID="b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f" exitCode=0
Sep 09 03:40:20.471192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.471006    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"
Sep 09 03:40:20.471192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.471015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz" event={"ID":"0d949391-ec9d-4153-8e91-c07ae408528b","Type":"ContainerDied","Data":"b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f"}
Sep 09 03:40:20.471192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.471050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz" event={"ID":"0d949391-ec9d-4153-8e91-c07ae408528b","Type":"ContainerDied","Data":"2ec2a0b441d1353ba5483fe76dafa72ce7cd3bf069a3e4564580cce3996e169d"}
Sep 09 03:40:20.471192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.471066    2580 scope.go:117] "RemoveContainer" containerID="b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f"
Sep 09 03:40:20.481354 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.481338    2580 scope.go:117] "RemoveContainer" containerID="b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f"
Sep 09 03:40:20.481633 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:40:20.481613    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f\": container with ID starting with b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f not found: ID does not exist" containerID="b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f"
Sep 09 03:40:20.481710 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.481646    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f"} err="failed to get container status \"b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f\": rpc error: code = NotFound desc = could not find container \"b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f\": container with ID starting with b47ab2ee35db6ff39f91531e172bff2b70dfe1c5a32cb7e1f8ff1bc1705cfe5f not found: ID does not exist"
Sep 09 03:40:20.498744 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.498721    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"]
Sep 09 03:40:20.501693 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:20.501659    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d8e85/sub-test-model-5d8e85-kserve-697475c48f-nc2qz"]
Sep 09 03:40:21.222138 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.222097    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb" podUID="ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" containerName="main" containerID="cri-o://d196c1dd99bb0bb16dde59de38bd917d95edb77cd36b1b38c15ce56f95da26bb" gracePeriod=45
Sep 09 03:40:21.480136 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.480105    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" containerID="d196c1dd99bb0bb16dde59de38bd917d95edb77cd36b1b38c15ce56f95da26bb" exitCode=0
Sep 09 03:40:21.480255 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.480175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb" event={"ID":"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975","Type":"ContainerDied","Data":"d196c1dd99bb0bb16dde59de38bd917d95edb77cd36b1b38c15ce56f95da26bb"}
Sep 09 03:40:21.480255 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.480213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb" event={"ID":"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975","Type":"ContainerDied","Data":"2d91ee698cacbc20f85c327c5079e209b8d713e314fd87fecc229faf7895ea04"}
Sep 09 03:40:21.480255 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.480224    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d91ee698cacbc20f85c327c5079e209b8d713e314fd87fecc229faf7895ea04"
Sep 09 03:40:21.490140 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.490123    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:40:21.579475 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-dshm\") pod \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") "
Sep 09 03:40:21.579650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579482    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tmp-dir\") pod \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") "
Sep 09 03:40:21.579650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579546    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-model-cache\") pod \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") "
Sep 09 03:40:21.579650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579592    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tls-certs\") pod \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") "
Sep 09 03:40:21.579650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbxb6\" (UniqueName: \"kubernetes.io/projected/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-kube-api-access-mbxb6\") pod \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") "
Sep 09 03:40:21.579893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-home\") pod \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\" (UID: \"ad3ab6c3-acbd-4878-a4c7-ef513ad1f975\") "
Sep 09 03:40:21.579893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579852    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" (UID: "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:21.579893 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.579871    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" (UID: "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:21.580063 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.580037    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-home" (OuterVolumeSpecName: "home") pod "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" (UID: "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:21.580164 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.580147    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:21.580164 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.580164    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:21.580385 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.580172    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:21.581914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.581888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-dshm" (OuterVolumeSpecName: "dshm") pod "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" (UID: "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:40:21.581914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.581888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" (UID: "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:40:21.582077 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.581978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-kube-api-access-mbxb6" (OuterVolumeSpecName: "kube-api-access-mbxb6") pod "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" (UID: "ad3ab6c3-acbd-4878-a4c7-ef513ad1f975"). InnerVolumeSpecName "kube-api-access-mbxb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:40:21.681469 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.681424    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:21.681469 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.681467    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbxb6\" (UniqueName: \"kubernetes.io/projected/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-kube-api-access-mbxb6\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:21.681760 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.681480    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:40:21.789996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:21.789923    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d949391-ec9d-4153-8e91-c07ae408528b" path="/var/lib/kubelet/pods/0d949391-ec9d-4153-8e91-c07ae408528b/volumes"
Sep 09 03:40:22.487939 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:22.487912    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"
Sep 09 03:40:22.507881 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:22.507846    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"]
Sep 09 03:40:22.511905 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:22.511882    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/sub-test-model-c34148-kserve-5d9dc8b6cc-jfbkb"]
Sep 09 03:40:23.789844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:23.789798    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" path="/var/lib/kubelet/pods/ad3ab6c3-acbd-4878-a4c7-ef513ad1f975/volumes"
Sep 09 03:40:31.549506 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:31.549475    2580 generic.go:356] "Generic (PLEG): container finished" podID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerID="ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54" exitCode=0
Sep 09 03:40:31.549918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:31.549537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" event={"ID":"dbd070c2-d40a-44f8-a5b3-7c6b88867385","Type":"ContainerDied","Data":"ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"}
Sep 09 03:40:31.549971 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:31.549956    2580 scope.go:117] "RemoveContainer" containerID="ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"
Sep 09 03:40:31.551292 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:31.551264    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:31.551365 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:31.551332    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:31.551404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:31.551364    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:40:32.560723 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:32.560688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" event={"ID":"dbd070c2-d40a-44f8-a5b3-7c6b88867385","Type":"ContainerStarted","Data":"e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694"}
Sep 09 03:40:33.381570 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:33.381536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:33.381797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:33.381772    2580 patch_prober.go:28] interesting pod/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 09 03:40:33.381851 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:33.381836    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 09 03:40:34.382354 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:34.382320    2580 patch_prober.go:28] interesting pod/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 09 03:40:34.382760 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:34.382403    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 09 03:40:35.382279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:35.382241    2580 patch_prober.go:28] interesting pod/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 09 03:40:35.382462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:35.382296    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 09 03:40:36.382173 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:36.382131    2580 patch_prober.go:28] interesting pod/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 09 03:40:36.382349 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:36.382193    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 09 03:40:37.381776 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:37.381744    2580 patch_prober.go:28] interesting pod/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 09 03:40:37.382175 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:37.381813    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 09 03:40:38.385646 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:38.385605    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:38.386147 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:38.386057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:40:38.386924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:40:38.386904    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:41:01.220791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:01.220748    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"]
Sep 09 03:41:01.221356 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:01.220999    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy" containerID="cri-o://ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1" gracePeriod=30
Sep 09 03:41:06.483068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.483011    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:41:06.504998 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.504961    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-credential-socket\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505135 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505033    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-token\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505135 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505059    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-certs\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505135 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505113    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-socket\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505313 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-podinfo\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505313 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knttd\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-kube-api-access-knttd\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505313 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505246    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7acda3a1-8730-4c53-8583-cb10f69cf41f-istiod-ca-cert\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505305    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:06.505464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-envoy\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505464 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-data\") pod \"7acda3a1-8730-4c53-8583-cb10f69cf41f\" (UID: \"7acda3a1-8730-4c53-8583-cb10f69cf41f\") "
Sep 09 03:41:06.505595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:06.505812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505791    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-data" (OuterVolumeSpecName: "istio-data") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:06.505896 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505864    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.505896 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.505886    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.506529 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.506057    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:06.506529 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.506075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7acda3a1-8730-4c53-8583-cb10f69cf41f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:06.509524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.509435    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:41:06.509524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.509447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-kube-api-access-knttd" (OuterVolumeSpecName: "kube-api-access-knttd") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "kube-api-access-knttd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:06.509524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.509508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-token" (OuterVolumeSpecName: "istio-token") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:06.509741 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.509543    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7acda3a1-8730-4c53-8583-cb10f69cf41f" (UID: "7acda3a1-8730-4c53-8583-cb10f69cf41f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:06.606448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606423    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7acda3a1-8730-4c53-8583-cb10f69cf41f-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.606448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606447    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.606574 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606456    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.606574 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606463    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.606574 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606470    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7acda3a1-8730-4c53-8583-cb10f69cf41f-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.606574 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606478    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7acda3a1-8730-4c53-8583-cb10f69cf41f-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.606574 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.606485    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knttd\" (UniqueName: \"kubernetes.io/projected/7acda3a1-8730-4c53-8583-cb10f69cf41f-kube-api-access-knttd\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:06.782709 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.782597    2580 generic.go:356] "Generic (PLEG): container finished" podID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerID="ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1" exitCode=0
Sep 09 03:41:06.782709 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.782641    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"
Sep 09 03:41:06.783072 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.782702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" event={"ID":"7acda3a1-8730-4c53-8583-cb10f69cf41f","Type":"ContainerDied","Data":"ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1"}
Sep 09 03:41:06.783072 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.782756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9" event={"ID":"7acda3a1-8730-4c53-8583-cb10f69cf41f","Type":"ContainerDied","Data":"c8294230bb8ccb765f493d3a8241e943bb944f2b25557eba3d511796f21bed26"}
Sep 09 03:41:06.783072 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.782778    2580 scope.go:117] "RemoveContainer" containerID="ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1"
Sep 09 03:41:06.793567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.793553    2580 scope.go:117] "RemoveContainer" containerID="4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"
Sep 09 03:41:06.802968 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.802953    2580 scope.go:117] "RemoveContainer" containerID="ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1"
Sep 09 03:41:06.803228 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:41:06.803208    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1\": container with ID starting with ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1 not found: ID does not exist" containerID="ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1"
Sep 09 03:41:06.803293 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.803234    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1"} err="failed to get container status \"ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1\": rpc error: code = NotFound desc = could not find container \"ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1\": container with ID starting with ccf0f6778a6820438e283706db8b11298418125584166f527266cbfc643f87f1 not found: ID does not exist"
Sep 09 03:41:06.803293 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.803249    2580 scope.go:117] "RemoveContainer" containerID="4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"
Sep 09 03:41:06.803477 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:41:06.803460    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642\": container with ID starting with 4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642 not found: ID does not exist" containerID="4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"
Sep 09 03:41:06.803517 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.803480    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642"} err="failed to get container status \"4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642\": rpc error: code = NotFound desc = could not find container \"4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642\": container with ID starting with 4ae91f91a3e40a5832e7363b3feccf160ef802598d4ff97e1a5fc300223cd642 not found: ID does not exist"
Sep 09 03:41:06.804058 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.804041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"]
Sep 09 03:41:06.808227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:06.808205    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7f0891-openshift-default-59c48f589d-hwlz9"]
Sep 09 03:41:07.792547 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:07.792518    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" path="/var/lib/kubelet/pods/7acda3a1-8730-4c53-8583-cb10f69cf41f/volumes"
Sep 09 03:41:15.408389 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408358    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"]
Sep 09 03:41:15.408848 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408834    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy"
Sep 09 03:41:15.408898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408850    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy"
Sep 09 03:41:15.408898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408869    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" containerName="main"
Sep 09 03:41:15.408898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408874    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" containerName="main"
Sep 09 03:41:15.408995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408899    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d949391-ec9d-4153-8e91-c07ae408528b" containerName="main"
Sep 09 03:41:15.408995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408908    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d949391-ec9d-4153-8e91-c07ae408528b" containerName="main"
Sep 09 03:41:15.408995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408972    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d949391-ec9d-4153-8e91-c07ae408528b" containerName="main"
Sep 09 03:41:15.408995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408984    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy"
Sep 09 03:41:15.408995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.408993    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad3ab6c3-acbd-4878-a4c7-ef513ad1f975" containerName="main"
Sep 09 03:41:15.414254 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.414225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.418412 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.418389    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"default-dockercfg-gwznt\""
Sep 09 03:41:15.418562 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.418456    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"openshift-service-ca.crt\""
Sep 09 03:41:15.418562 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.418521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"rate-test-model-c34148-kserve-self-signed-certs\""
Sep 09 03:41:15.418712 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.418628    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c34148\"/\"kube-root-ca.crt\""
Sep 09 03:41:15.424685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.424646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"]
Sep 09 03:41:15.481968 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.481939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-home\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.482106 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.481991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-model-cache\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.482106 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.482044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-dshm\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.482106 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.482082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2fjx\" (UniqueName: \"kubernetes.io/projected/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-kube-api-access-r2fjx\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.482224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.482123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tmp-dir\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.482224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.482139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tls-certs\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.583413 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-model-cache\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.583544 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-dshm\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.583544 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fjx\" (UniqueName: \"kubernetes.io/projected/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-kube-api-access-r2fjx\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.583544 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tmp-dir\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.583544 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583497    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tls-certs\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.584302 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-home\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.584302 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-model-cache\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.584302 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.583895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-home\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.584302 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.584001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tmp-dir\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.593975 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.593940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-dshm\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.594194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.594175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tls-certs\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.596184 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.596167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fjx\" (UniqueName: \"kubernetes.io/projected/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-kube-api-access-r2fjx\") pod \"rate-test-model-c34148-kserve-5fc6c897ff-8q5js\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") " pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:15.727336 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:15.727307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:16.002283 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:16.002256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"]
Sep 09 03:41:16.003568 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:41:16.003533    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6737bf3_1819_4a53_950c_9ddc79e6d1f8.slice/crio-5492c7773a131f9b2839a3e00e66f729d8ec8398b8fa4efea72ffe220ad89417 WatchSource:0}: Error finding container 5492c7773a131f9b2839a3e00e66f729d8ec8398b8fa4efea72ffe220ad89417: Status 404 returned error can't find the container with id 5492c7773a131f9b2839a3e00e66f729d8ec8398b8fa4efea72ffe220ad89417
Sep 09 03:41:16.848561 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:16.848533    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js" event={"ID":"c6737bf3-1819-4a53-950c-9ddc79e6d1f8","Type":"ContainerStarted","Data":"3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e"}
Sep 09 03:41:16.848969 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:16.848574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js" event={"ID":"c6737bf3-1819-4a53-950c-9ddc79e6d1f8","Type":"ContainerStarted","Data":"5492c7773a131f9b2839a3e00e66f729d8ec8398b8fa4efea72ffe220ad89417"}
Sep 09 03:41:16.867218 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:16.867172    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js" podStartSLOduration=1.867158024 podStartE2EDuration="1.867158024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:41:16.866298711 +0000 UTC m=+1687.566062671" watchObservedRunningTime="2026-09-09 03:41:16.867158024 +0000 UTC m=+1687.566921985"
Sep 09 03:41:24.139512 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:24.139472    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"]
Sep 09 03:41:24.140135 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:24.139772    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy" containerID="cri-o://e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694" gracePeriod=30
Sep 09 03:41:25.727900 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:25.727868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:25.728363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:25.727918    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:25.740514 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:25.740492    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:25.913492 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:25.913467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:41:29.391342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.391318    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:41:29.512494 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-data\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-credential-socket\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512569    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-socket\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512604    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-certs\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512650 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512639    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-token\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsn59\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-kube-api-access-nsn59\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512751    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-podinfo\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:29.512880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-envoy\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.512880 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istiod-ca-cert\") pod \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\" (UID: \"dbd070c2-d40a-44f8-a5b3-7c6b88867385\") "
Sep 09 03:41:29.513164 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512889    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:29.513164 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.512984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:29.513265 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.513186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:29.513320 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.513280    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:29.513320 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.513283    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.513320 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.513319    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.513474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.513334    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.513474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.513348    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.515039 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.515016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:29.515167 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.515069    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:29.515387 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.515370    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:41:29.515454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.515377    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-kube-api-access-nsn59" (OuterVolumeSpecName: "kube-api-access-nsn59") pod "dbd070c2-d40a-44f8-a5b3-7c6b88867385" (UID: "dbd070c2-d40a-44f8-a5b3-7c6b88867385"). InnerVolumeSpecName "kube-api-access-nsn59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:29.614085 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.614064    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.614085 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.614086    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsn59\" (UniqueName: \"kubernetes.io/projected/dbd070c2-d40a-44f8-a5b3-7c6b88867385-kube-api-access-nsn59\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.614243 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.614097    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.614243 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.614106    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.614243 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.614113    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbd070c2-d40a-44f8-a5b3-7c6b88867385-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:29.932754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.932719    2580 generic.go:356] "Generic (PLEG): container finished" podID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerID="e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694" exitCode=0
Sep 09 03:41:29.932906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.932770    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"
Sep 09 03:41:29.932906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.932801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" event={"ID":"dbd070c2-d40a-44f8-a5b3-7c6b88867385","Type":"ContainerDied","Data":"e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694"}
Sep 09 03:41:29.932906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.932865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr" event={"ID":"dbd070c2-d40a-44f8-a5b3-7c6b88867385","Type":"ContainerDied","Data":"306429e4016a29a53d19db721231fa56c3f15607d250bcadbee0707c37e8b072"}
Sep 09 03:41:29.932906 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.932898    2580 scope.go:117] "RemoveContainer" containerID="e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694"
Sep 09 03:41:29.943069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.943053    2580 scope.go:117] "RemoveContainer" containerID="ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"
Sep 09 03:41:29.953199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.953180    2580 scope.go:117] "RemoveContainer" containerID="e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694"
Sep 09 03:41:29.953498 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:41:29.953473    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694\": container with ID starting with e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694 not found: ID does not exist" containerID="e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694"
Sep 09 03:41:29.953567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.953509    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694"} err="failed to get container status \"e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694\": rpc error: code = NotFound desc = could not find container \"e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694\": container with ID starting with e3052dd6037503c0bbbac077b3c2ea7fe99f870ec4e2023a9ccd4c5ac2c88694 not found: ID does not exist"
Sep 09 03:41:29.953567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.953533    2580 scope.go:117] "RemoveContainer" containerID="ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"
Sep 09 03:41:29.953858 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:41:29.953840    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54\": container with ID starting with ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54 not found: ID does not exist" containerID="ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"
Sep 09 03:41:29.953915 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.953865    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54"} err="failed to get container status \"ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54\": rpc error: code = NotFound desc = could not find container \"ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54\": container with ID starting with ded39135c02b7c55812d6a75908b8c9068af244317e2e54778423a647833ae54 not found: ID does not exist"
Sep 09 03:41:29.955600 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.955583    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"]
Sep 09 03:41:29.961587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:29.961567    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1c4cb1-openshift-default-676c65f6b6-wlcmr"]
Sep 09 03:41:31.793620 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:31.793413    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" path="/var/lib/kubelet/pods/dbd070c2-d40a-44f8-a5b3-7c6b88867385/volumes"
Sep 09 03:41:41.871837 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:41.871802    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"]
Sep 09 03:41:41.872337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:41.872043    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" containerID="cri-o://38f2bf17c1d3c850992d3fffc95c8ee99eb4508ea5d84e24455df313d1d3df0d" gracePeriod=30
Sep 09 03:41:44.853422 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:44.853387    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:41:44.853810 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:44.853441    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:41:47.040642 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.040607    2580 generic.go:356] "Generic (PLEG): container finished" podID="e49ee436-2a31-4741-bc26-693703b1fa69" containerID="38f2bf17c1d3c850992d3fffc95c8ee99eb4508ea5d84e24455df313d1d3df0d" exitCode=0
Sep 09 03:41:47.041118 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.040704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" event={"ID":"e49ee436-2a31-4741-bc26-693703b1fa69","Type":"ContainerDied","Data":"38f2bf17c1d3c850992d3fffc95c8ee99eb4508ea5d84e24455df313d1d3df0d"}
Sep 09 03:41:47.041118 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.040766    2580 scope.go:117] "RemoveContainer" containerID="ec66287ca832353df3c9cbffd96a8fc62d1def8d5375b06e2a464bf550e19360"
Sep 09 03:41:47.129137 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.129110    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:41:47.178821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.178749    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-socket\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.178821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.178809    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sr9f\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-kube-api-access-6sr9f\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.178829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-data\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.178848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-credential-socket\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.178887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee436-2a31-4741-bc26-693703b1fa69-istiod-ca-cert\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.178931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49ee436-2a31-4741-bc26-693703b1fa69-istio-podinfo\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179024    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:47.179295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179043    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-istio-token\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179100    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-envoy\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179128    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-certs\") pod \"e49ee436-2a31-4741-bc26-693703b1fa69\" (UID: \"e49ee436-2a31-4741-bc26-693703b1fa69\") "
Sep 09 03:41:47.179295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-data" (OuterVolumeSpecName: "istio-data") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:47.179480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179312    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:47.179571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179549    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.179635 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179578    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.179635 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179584    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49ee436-2a31-4741-bc26-693703b1fa69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:41:47.179635 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179591    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.179793 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.179660    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:47.181484 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.181451    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-istio-token" (OuterVolumeSpecName: "istio-token") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:47.181732 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.181698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:41:47.181837 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.181793    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-kube-api-access-6sr9f" (OuterVolumeSpecName: "kube-api-access-6sr9f") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "kube-api-access-6sr9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:41:47.181837 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.181824    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e49ee436-2a31-4741-bc26-693703b1fa69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e49ee436-2a31-4741-bc26-693703b1fa69" (UID: "e49ee436-2a31-4741-bc26-693703b1fa69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:41:47.280519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.280488    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49ee436-2a31-4741-bc26-693703b1fa69-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.280519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.280514    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49ee436-2a31-4741-bc26-693703b1fa69-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.280519 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.280522    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.280758 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.280531    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.280758 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.280541    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49ee436-2a31-4741-bc26-693703b1fa69-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:47.280758 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:47.280549    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sr9f\" (UniqueName: \"kubernetes.io/projected/e49ee436-2a31-4741-bc26-693703b1fa69-kube-api-access-6sr9f\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:41:48.048510 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:48.048423    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"
Sep 09 03:41:48.048510 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:48.048459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl" event={"ID":"e49ee436-2a31-4741-bc26-693703b1fa69","Type":"ContainerDied","Data":"ce3878e54c3e6a19eb5608c9289a190b76b91b0075a0739a637e8169cbc288da"}
Sep 09 03:41:48.049021 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:48.048524    2580 scope.go:117] "RemoveContainer" containerID="38f2bf17c1d3c850992d3fffc95c8ee99eb4508ea5d84e24455df313d1d3df0d"
Sep 09 03:41:48.068090 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:48.068062    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"]
Sep 09 03:41:48.074556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:48.074535    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-eca017-openshift-default-69cff558b4-jc6rl"]
Sep 09 03:41:49.790321 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:49.790291    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" path="/var/lib/kubelet/pods/e49ee436-2a31-4741-bc26-693703b1fa69/volumes"
Sep 09 03:41:59.635332 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:59.635285    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"]
Sep 09 03:41:59.635851 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:41:59.635515    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" containerID="cri-o://5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d" gracePeriod=30
Sep 09 03:42:00.359300 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:00.359268    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:42:00.359466 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:00.359318    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:42:04.910534 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:04.910462    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:42:05.057628 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057598    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-socket\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.057628 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057627    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-certs\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.057863 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-data\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.057863 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057762    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-podinfo\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.057863 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-token\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.057863 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057823    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-envoy\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.058088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istiod-ca-cert\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.058088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-credential-socket\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.058088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057889    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:05.058088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.057963    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:05.058088 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058014    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qhr2\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-kube-api-access-8qhr2\") pod \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\" (UID: \"c41d7cdc-57d2-427d-ba6d-4260c30a3379\") "
Sep 09 03:42:05.058362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058145    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:05.058362 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058309    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:42:05.058482 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058383    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.058482 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058402    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.058482 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058417    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.058482 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.058398    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-data" (OuterVolumeSpecName: "istio-data") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:05.060326 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.060276    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:05.060509 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.060377    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:42:05.060571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.060509    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-kube-api-access-8qhr2" (OuterVolumeSpecName: "kube-api-access-8qhr2") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "kube-api-access-8qhr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:42:05.060788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.060767    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-token" (OuterVolumeSpecName: "istio-token") pod "c41d7cdc-57d2-427d-ba6d-4260c30a3379" (UID: "c41d7cdc-57d2-427d-ba6d-4260c30a3379"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:42:05.158949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.158919    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.158949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.158941    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.158949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.158950    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.158949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.158958    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.159226 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.158966    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c41d7cdc-57d2-427d-ba6d-4260c30a3379-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.159226 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.158973    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qhr2\" (UniqueName: \"kubernetes.io/projected/c41d7cdc-57d2-427d-ba6d-4260c30a3379-kube-api-access-8qhr2\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:05.164109 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.164031    2580 generic.go:356] "Generic (PLEG): container finished" podID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerID="5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d" exitCode=0
Sep 09 03:42:05.164109 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.164086    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"
Sep 09 03:42:05.164109 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.164077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" event={"ID":"c41d7cdc-57d2-427d-ba6d-4260c30a3379","Type":"ContainerDied","Data":"5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d"}
Sep 09 03:42:05.164350 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.164153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f" event={"ID":"c41d7cdc-57d2-427d-ba6d-4260c30a3379","Type":"ContainerDied","Data":"fc80f37a16cbe4d76ef8dcbb94131c92785e059a4095d7a8614b8f8c986575d7"}
Sep 09 03:42:05.164350 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.164176    2580 scope.go:117] "RemoveContainer" containerID="5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d"
Sep 09 03:42:05.174626 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.174611    2580 scope.go:117] "RemoveContainer" containerID="e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"
Sep 09 03:42:05.184086 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.184071    2580 scope.go:117] "RemoveContainer" containerID="5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d"
Sep 09 03:42:05.184325 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:42:05.184305    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d\": container with ID starting with 5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d not found: ID does not exist" containerID="5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d"
Sep 09 03:42:05.184400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.184339    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d"} err="failed to get container status \"5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d\": rpc error: code = NotFound desc = could not find container \"5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d\": container with ID starting with 5cc72898cccb2432f12b6da34035955be57c2de24ec833b55e184cfbe313664d not found: ID does not exist"
Sep 09 03:42:05.184400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.184364    2580 scope.go:117] "RemoveContainer" containerID="e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"
Sep 09 03:42:05.184643 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:42:05.184626    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a\": container with ID starting with e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a not found: ID does not exist" containerID="e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"
Sep 09 03:42:05.184707 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.184648    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a"} err="failed to get container status \"e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a\": rpc error: code = NotFound desc = could not find container \"e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a\": container with ID starting with e1909bbb9dad2a5ad06a1cd1c8999aa3d7de461b18a67bdb39136103773ae70a not found: ID does not exist"
Sep 09 03:42:05.199018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.198995    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"]
Sep 09 03:42:05.214663 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.214633    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0d0cae-openshift-default-8ccd995bb-px97f"]
Sep 09 03:42:05.789928 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:05.789898    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" path="/var/lib/kubelet/pods/c41d7cdc-57d2-427d-ba6d-4260c30a3379/volumes"
Sep 09 03:42:10.042591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:10.042566    2580 scope.go:117] "RemoveContainer" containerID="93a65fd338c1efe2cd17fe7ace2f2370dbb61c67a973c967cd5fd9349d6e3531"
Sep 09 03:42:10.051636 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:10.051616    2580 scope.go:117] "RemoveContainer" containerID="458d42de9a3e4e1b8a7ba53d35afefda2c5f90153d7fcd25abc8fe168d45c428"
Sep 09 03:42:11.799347 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:11.799314    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"]
Sep 09 03:42:12.562425 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562392    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"]
Sep 09 03:42:12.562897 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562880    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562899    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562915    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562923    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562942    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562950    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562973    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562981    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.562995    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy"
Sep 09 03:42:12.563010 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563003    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563018    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563027    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563129    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563140    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41d7cdc-57d2-427d-ba6d-4260c30a3379" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563151    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563162    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7acda3a1-8730-4c53-8583-cb10f69cf41f" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563170    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbd070c2-d40a-44f8-a5b3-7c6b88867385" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563179    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy"
Sep 09 03:42:12.563446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.563188    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy"
Sep 09 03:42:12.566398 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.566379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.569820 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.569804    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-46be70-openshift-default-dockercfg-v8c9c\""
Sep 09 03:42:12.581425 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.581402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"]
Sep 09 03:42:12.729417 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-socket\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729417 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-envoy\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-podinfo\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-credential-socket\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d786e82e-32e5-4792-9fdb-58eda90cd466-istiod-ca-cert\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-token\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d94f\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-kube-api-access-7d94f\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729630 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-certs\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.729866 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.729708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-data\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.830924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.830857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-data\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.830924 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.830907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-socket\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-envoy\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-podinfo\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-credential-socket\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d786e82e-32e5-4792-9fdb-58eda90cd466-istiod-ca-cert\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-token\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d94f\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-kube-api-access-7d94f\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-certs\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-socket\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831404 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-data\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-credential-socket\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831876 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-certs\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.831989 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.831882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d786e82e-32e5-4792-9fdb-58eda90cd466-istiod-ca-cert\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.833917 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.833885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-envoy\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.834211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.834190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-podinfo\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.839908 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.839882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-token\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.840390 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.840373    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d94f\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-kube-api-access-7d94f\") pod \"e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") " pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:12.877068 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:12.877044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:13.160504 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:13.160477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"]
Sep 09 03:42:13.161406 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:42:13.161378    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd786e82e_32e5_4792_9fdb_58eda90cd466.slice/crio-58f178ad4340b430d1d4a7d0b8d86dd3461fdeba6ef073d93bccc6df31563ebb WatchSource:0}: Error finding container 58f178ad4340b430d1d4a7d0b8d86dd3461fdeba6ef073d93bccc6df31563ebb: Status 404 returned error can't find the container with id 58f178ad4340b430d1d4a7d0b8d86dd3461fdeba6ef073d93bccc6df31563ebb
Sep 09 03:42:13.163800 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:13.163762    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:13.163910 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:13.163841    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:13.163910 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:13.163877    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:13.214190 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:13.214164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" event={"ID":"d786e82e-32e5-4792-9fdb-58eda90cd466","Type":"ContainerStarted","Data":"58f178ad4340b430d1d4a7d0b8d86dd3461fdeba6ef073d93bccc6df31563ebb"}
Sep 09 03:42:14.221759 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:14.221723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" event={"ID":"d786e82e-32e5-4792-9fdb-58eda90cd466","Type":"ContainerStarted","Data":"a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"}
Sep 09 03:42:14.245127 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:14.245077    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podStartSLOduration=2.245062195 podStartE2EDuration="2.245062195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:42:14.242627472 +0000 UTC m=+1744.942391434" watchObservedRunningTime="2026-09-09 03:42:14.245062195 +0000 UTC m=+1744.944826156"
Sep 09 03:42:14.877703 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:14.877654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:14.883190 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:14.883167    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:15.227476 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:15.227442    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:15.228471 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:15.228443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:26.819303 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:26.819259    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js" podUID="c6737bf3-1819-4a53-950c-9ddc79e6d1f8" containerName="main" containerID="cri-o://3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e" gracePeriod=45
Sep 09 03:42:27.106792 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.106763    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:42:27.176363 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176332    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2fjx\" (UniqueName: \"kubernetes.io/projected/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-kube-api-access-r2fjx\") pod \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") "
Sep 09 03:42:27.176527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176400    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-model-cache\") pod \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") "
Sep 09 03:42:27.176527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176431    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-home\") pod \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") "
Sep 09 03:42:27.176527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176449    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tls-certs\") pod \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") "
Sep 09 03:42:27.176527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176478    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tmp-dir\") pod \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") "
Sep 09 03:42:27.176527 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176514    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-dshm\") pod \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\" (UID: \"c6737bf3-1819-4a53-950c-9ddc79e6d1f8\") "
Sep 09 03:42:27.176830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176649    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-home" (OuterVolumeSpecName: "home") pod "c6737bf3-1819-4a53-950c-9ddc79e6d1f8" (UID: "c6737bf3-1819-4a53-950c-9ddc79e6d1f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:27.176830 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176791    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:27.176942 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6737bf3-1819-4a53-950c-9ddc79e6d1f8" (UID: "c6737bf3-1819-4a53-950c-9ddc79e6d1f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:27.177017 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.176992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6737bf3-1819-4a53-950c-9ddc79e6d1f8" (UID: "c6737bf3-1819-4a53-950c-9ddc79e6d1f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:27.178895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.178869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6737bf3-1819-4a53-950c-9ddc79e6d1f8" (UID: "c6737bf3-1819-4a53-950c-9ddc79e6d1f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:42:27.179026 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.178955    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-kube-api-access-r2fjx" (OuterVolumeSpecName: "kube-api-access-r2fjx") pod "c6737bf3-1819-4a53-950c-9ddc79e6d1f8" (UID: "c6737bf3-1819-4a53-950c-9ddc79e6d1f8"). InnerVolumeSpecName "kube-api-access-r2fjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:42:27.179117 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.179100    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-dshm" (OuterVolumeSpecName: "dshm") pod "c6737bf3-1819-4a53-950c-9ddc79e6d1f8" (UID: "c6737bf3-1819-4a53-950c-9ddc79e6d1f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:42:27.277996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.277966    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:27.277996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.277992    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:27.277996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.278000    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:27.278211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.278009    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2fjx\" (UniqueName: \"kubernetes.io/projected/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-kube-api-access-r2fjx\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:27.278211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.278020    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6737bf3-1819-4a53-950c-9ddc79e6d1f8-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:42:27.297390 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.297357    2580 generic.go:356] "Generic (PLEG): container finished" podID="c6737bf3-1819-4a53-950c-9ddc79e6d1f8" containerID="3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e" exitCode=0
Sep 09 03:42:27.297490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.297402    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"
Sep 09 03:42:27.297490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.297426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js" event={"ID":"c6737bf3-1819-4a53-950c-9ddc79e6d1f8","Type":"ContainerDied","Data":"3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e"}
Sep 09 03:42:27.297490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.297463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js" event={"ID":"c6737bf3-1819-4a53-950c-9ddc79e6d1f8","Type":"ContainerDied","Data":"5492c7773a131f9b2839a3e00e66f729d8ec8398b8fa4efea72ffe220ad89417"}
Sep 09 03:42:27.297490 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.297483    2580 scope.go:117] "RemoveContainer" containerID="3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e"
Sep 09 03:42:27.307416 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.307399    2580 scope.go:117] "RemoveContainer" containerID="3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e"
Sep 09 03:42:27.307717 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:42:27.307698    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e\": container with ID starting with 3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e not found: ID does not exist" containerID="3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e"
Sep 09 03:42:27.307783 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.307726    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e"} err="failed to get container status \"3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e\": rpc error: code = NotFound desc = could not find container \"3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e\": container with ID starting with 3b33994bbc75960b0f8a405121e92461cb61ee3cf360745e25f3d5fe6af8a72e not found: ID does not exist"
Sep 09 03:42:27.320283 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.320260    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"]
Sep 09 03:42:27.343745 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.327071    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c34148/rate-test-model-c34148-kserve-5fc6c897ff-8q5js"]
Sep 09 03:42:27.789610 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:27.789569    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6737bf3-1819-4a53-950c-9ddc79e6d1f8" path="/var/lib/kubelet/pods/c6737bf3-1819-4a53-950c-9ddc79e6d1f8/volumes"
Sep 09 03:42:31.268800 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:31.268771    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 03:42:31.269192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:31.268841    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 03:42:32.329860 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:32.329826    2580 generic.go:356] "Generic (PLEG): container finished" podID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerID="a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53" exitCode=0
Sep 09 03:42:32.330384 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:32.329908    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" event={"ID":"d786e82e-32e5-4792-9fdb-58eda90cd466","Type":"ContainerDied","Data":"a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"}
Sep 09 03:42:32.330457 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:32.330411    2580 scope.go:117] "RemoveContainer" containerID="a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"
Sep 09 03:42:32.331474 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:32.331441    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:32.331590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:32.331534    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:32.331590 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:32.331582    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:33.344246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:33.344199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" event={"ID":"d786e82e-32e5-4792-9fdb-58eda90cd466","Type":"ContainerStarted","Data":"4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf"}
Sep 09 03:42:33.877281 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:33.877237    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:33.877529 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:33.877502    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 09 03:42:33.877658 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:33.877552    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 09 03:42:34.878091 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:34.878055    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 09 03:42:34.878475 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:34.878123    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 09 03:42:35.877696 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:35.877633    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 09 03:42:35.877911 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:35.877737    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 09 03:42:36.878358 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:36.878330    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 09 03:42:36.878767 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:36.878378    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 09 03:42:37.877722 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:37.877666    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 09 03:42:37.877895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:37.877757    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 09 03:42:38.881124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:38.881096    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:38.881571 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:38.881552    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:38.882290 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:38.882269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:42:42.287109 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.287068    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"]
Sep 09 03:42:42.287589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.287479    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6737bf3-1819-4a53-950c-9ddc79e6d1f8" containerName="main"
Sep 09 03:42:42.287589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.287490    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6737bf3-1819-4a53-950c-9ddc79e6d1f8" containerName="main"
Sep 09 03:42:42.287589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.287506    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy"
Sep 09 03:42:42.287589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.287512    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49ee436-2a31-4741-bc26-693703b1fa69" containerName="istio-proxy"
Sep 09 03:42:42.287589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.287588    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6737bf3-1819-4a53-950c-9ddc79e6d1f8" containerName="main"
Sep 09 03:42:42.290766 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.290744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.293366 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.293347    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-60a731-openshift-default-dockercfg-5dxl5\""
Sep 09 03:42:42.307160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.307135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"]
Sep 09 03:42:42.426042 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-data\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426235 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-envoy\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426235 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-socket\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426374 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-podinfo\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426374 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-token\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-credential-socket\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istiod-ca-cert\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zstrg\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-kube-api-access-zstrg\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.426495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.426472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-certs\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-socket\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527392 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-podinfo\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-token\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-credential-socket\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istiod-ca-cert\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstrg\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-kube-api-access-zstrg\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-certs\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527652 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-data\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527977 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-envoy\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.527977 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.527811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-socket\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.528110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.528079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-certs\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.528228 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.528184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-data\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.528545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.528285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istiod-ca-cert\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.528545 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.528347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-credential-socket\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.530029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.530008    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-envoy\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.530454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.530435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-podinfo\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.536225 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.536203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-token\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.536343 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.536325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zstrg\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-kube-api-access-zstrg\") pod \"e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") " pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.602891 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.602833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:42.881153 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.881123    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"]
Sep 09 03:42:42.881786 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:42:42.881761    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod064a51ff_d3fc_4feb_928a_f9cce6e2c437.slice/crio-ef7a856aac2f584474a0393b8e15a5f273e5a8332988dd02dfe8b04ccc00a01d WatchSource:0}: Error finding container ef7a856aac2f584474a0393b8e15a5f273e5a8332988dd02dfe8b04ccc00a01d: Status 404 returned error can't find the container with id ef7a856aac2f584474a0393b8e15a5f273e5a8332988dd02dfe8b04ccc00a01d
Sep 09 03:42:42.883997 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.883951    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:42.884076 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.884048    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:42.884118 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:42.884097    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:42:43.415013 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:43.414982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" event={"ID":"064a51ff-d3fc-4feb-928a-f9cce6e2c437","Type":"ContainerStarted","Data":"d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"}
Sep 09 03:42:43.415420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:43.415042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" event={"ID":"064a51ff-d3fc-4feb-928a-f9cce6e2c437","Type":"ContainerStarted","Data":"ef7a856aac2f584474a0393b8e15a5f273e5a8332988dd02dfe8b04ccc00a01d"}
Sep 09 03:42:43.436136 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:43.436090    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podStartSLOduration=1.4360760909999999 podStartE2EDuration="1.436076091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:42:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:42:43.432897033 +0000 UTC m=+1774.132661028" watchObservedRunningTime="2026-09-09 03:42:43.436076091 +0000 UTC m=+1774.135840052"
Sep 09 03:42:43.603666 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:43.603633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:43.608983 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:43.608959    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:44.421595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:44.421564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:42:44.422451 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:42:44.422431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:43:02.535532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:02.535499    2580 generic.go:356] "Generic (PLEG): container finished" podID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerID="d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f" exitCode=0
Sep 09 03:43:02.535933 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:02.535571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" event={"ID":"064a51ff-d3fc-4feb-928a-f9cce6e2c437","Type":"ContainerDied","Data":"d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"}
Sep 09 03:43:02.536018 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:02.536005    2580 scope.go:117] "RemoveContainer" containerID="d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"
Sep 09 03:43:02.537179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:02.537149    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:43:02.537240 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:02.537221    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:43:02.537279 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:02.537253    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:43:03.549337 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:03.548362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" event={"ID":"064a51ff-d3fc-4feb-928a-f9cce6e2c437","Type":"ContainerStarted","Data":"2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083"}
Sep 09 03:43:03.603520 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:03.603486    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:43:03.603752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:03.603728    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 09 03:43:03.603846 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:03.603797    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 09 03:43:04.603755 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:04.603720    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 09 03:43:04.604159 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:04.603787    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 09 03:43:05.603233 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:05.603204    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 09 03:43:05.603412 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:05.603254    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 09 03:43:06.603815 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:06.603783    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 09 03:43:06.604201 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:06.603840    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 09 03:43:07.603929 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:07.603894    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 09 03:43:07.604315 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:07.603951    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 09 03:43:08.778453 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:08.778418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:43:08.778972 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:08.778849    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:43:08.779620 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:08.779598    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:43:21.088599 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.088560    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"]
Sep 09 03:43:21.091311 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.091289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.094725 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.094662    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-46be70\"/\"kube-root-ca.crt\""
Sep 09 03:43:21.094853 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.094736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-46be70\"/\"default-dockercfg-7kcz5\""
Sep 09 03:43:21.094853 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.094693    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-46be70\"/\"ipp-route-46be70-kserve-self-signed-certs\""
Sep 09 03:43:21.094853 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.094695    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-46be70\"/\"openshift-service-ca.crt\""
Sep 09 03:43:21.101579 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.101556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"]
Sep 09 03:43:21.187996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.187971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-dshm\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.188110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.188028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-tmp-dir\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.188110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.188057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0463ff49-c9fb-470f-a7c8-41c200173422-tls-certs\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.188110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.188075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-home\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.188110 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.188098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-model-cache\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.188250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.188115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmtz\" (UniqueName: \"kubernetes.io/projected/0463ff49-c9fb-470f-a7c8-41c200173422-kube-api-access-vcmtz\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289429 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-dshm\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-tmp-dir\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289611 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0463ff49-c9fb-470f-a7c8-41c200173422-tls-certs\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289656 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-home\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289735 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-model-cache\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289735 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmtz\" (UniqueName: \"kubernetes.io/projected/0463ff49-c9fb-470f-a7c8-41c200173422-kube-api-access-vcmtz\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289848 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-tmp-dir\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.289986 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.289965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-home\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.290046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.290018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-model-cache\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.291790 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.291762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-dshm\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.292071 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.292057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0463ff49-c9fb-470f-a7c8-41c200173422-tls-certs\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.298803 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.298778    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmtz\" (UniqueName: \"kubernetes.io/projected/0463ff49-c9fb-470f-a7c8-41c200173422-kube-api-access-vcmtz\") pod \"ipp-route-46be70-kserve-57df4c8587-7rzdk\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") " pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.401965 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.401880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:21.682194 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:21.682154    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"]
Sep 09 03:43:21.683014 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:43:21.682977    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0463ff49_c9fb_470f_a7c8_41c200173422.slice/crio-354c33197f42696846d3141d3b272bb335673d4e20ebe769bd0fa4ac7fd8d0a7 WatchSource:0}: Error finding container 354c33197f42696846d3141d3b272bb335673d4e20ebe769bd0fa4ac7fd8d0a7: Status 404 returned error can't find the container with id 354c33197f42696846d3141d3b272bb335673d4e20ebe769bd0fa4ac7fd8d0a7
Sep 09 03:43:22.678465 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:22.678421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk" event={"ID":"0463ff49-c9fb-470f-a7c8-41c200173422","Type":"ContainerStarted","Data":"18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0"}
Sep 09 03:43:22.678868 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:22.678475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk" event={"ID":"0463ff49-c9fb-470f-a7c8-41c200173422","Type":"ContainerStarted","Data":"354c33197f42696846d3141d3b272bb335673d4e20ebe769bd0fa4ac7fd8d0a7"}
Sep 09 03:43:22.697503 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:22.697454    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk" podStartSLOduration=1.6974389680000002 podStartE2EDuration="1.697438968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:43:22.69441867 +0000 UTC m=+1813.394182632" watchObservedRunningTime="2026-09-09 03:43:22.697438968 +0000 UTC m=+1813.397202931"
Sep 09 03:43:31.403029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:31.402993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:31.403420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:31.403043    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:31.415213 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:31.415186    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:31.745159 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:31.745135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:43:52.479514 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.479484    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"]
Sep 09 03:43:52.482470 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.482446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.485507 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.485487    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-65a459-openshift-default-dockercfg-mk5h6\""
Sep 09 03:43:52.495275 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.495253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"]
Sep 09 03:43:52.566414 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-socket\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm4zj\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-kube-api-access-qm4zj\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-data\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-credential-socket\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566614 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90abf08f-f646-4529-98d4-6b3d019a3454-istio-podinfo\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-istio-token\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90abf08f-f646-4529-98d4-6b3d019a3454-istiod-ca-cert\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-certs\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.566878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.566801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-envoy\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.667706 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-socket\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.667899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm4zj\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-kube-api-access-qm4zj\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.667899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-data\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.667899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-credential-socket\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.667899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90abf08f-f646-4529-98d4-6b3d019a3454-istio-podinfo\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.667899 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-istio-token\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90abf08f-f646-4529-98d4-6b3d019a3454-istiod-ca-cert\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-certs\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.667998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-envoy\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.668093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-socket\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.668147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-data\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.668221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-credential-socket\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668432 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.668306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-certs\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.668814 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.668795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90abf08f-f646-4529-98d4-6b3d019a3454-istiod-ca-cert\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.670400 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.670377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-envoy\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.670769 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.670749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90abf08f-f646-4529-98d4-6b3d019a3454-istio-podinfo\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.682721 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.682664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-istio-token\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.682812 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.682726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm4zj\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-kube-api-access-qm4zj\") pod \"e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") " pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:52.793771 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:52.793662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:53.072208 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.072182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"]
Sep 09 03:43:53.073439 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:43:53.073415    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90abf08f_f646_4529_98d4_6b3d019a3454.slice/crio-615a185e6f9e3a21fc5976fc3e6a35c423c71a8d90af81036c9e3bff9a916641 WatchSource:0}: Error finding container 615a185e6f9e3a21fc5976fc3e6a35c423c71a8d90af81036c9e3bff9a916641: Status 404 returned error can't find the container with id 615a185e6f9e3a21fc5976fc3e6a35c423c71a8d90af81036c9e3bff9a916641
Sep 09 03:43:53.075735 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.075666    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:43:53.075844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.075781    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:43:53.075844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.075826    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:43:53.872155 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.872122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" event={"ID":"90abf08f-f646-4529-98d4-6b3d019a3454","Type":"ContainerStarted","Data":"48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"}
Sep 09 03:43:53.872576 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.872174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" event={"ID":"90abf08f-f646-4529-98d4-6b3d019a3454","Type":"ContainerStarted","Data":"615a185e6f9e3a21fc5976fc3e6a35c423c71a8d90af81036c9e3bff9a916641"}
Sep 09 03:43:53.894873 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:53.894829    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podStartSLOduration=1.894814706 podStartE2EDuration="1.894814706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:43:53.893416361 +0000 UTC m=+1844.593180326" watchObservedRunningTime="2026-09-09 03:43:53.894814706 +0000 UTC m=+1844.594578667"
Sep 09 03:43:54.793911 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:54.793871    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:54.799473 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:54.799443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:54.877853 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:54.877831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:43:54.878774 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:43:54.878758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:15.015211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:15.015178    2580 generic.go:356] "Generic (PLEG): container finished" podID="90abf08f-f646-4529-98d4-6b3d019a3454" containerID="48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f" exitCode=0
Sep 09 03:44:15.015555 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:15.015248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" event={"ID":"90abf08f-f646-4529-98d4-6b3d019a3454","Type":"ContainerDied","Data":"48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"}
Sep 09 03:44:15.015615 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:15.015602    2580 scope.go:117] "RemoveContainer" containerID="48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"
Sep 09 03:44:15.016791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:15.016766    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:44:15.016867 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:15.016831    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:44:15.016907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:15.016864    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 03:44:16.023375 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:16.023343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" event={"ID":"90abf08f-f646-4529-98d4-6b3d019a3454","Type":"ContainerStarted","Data":"a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015"}
Sep 09 03:44:16.794954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:16.794920    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:16.794954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:16.794950    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 09 03:44:16.795185 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:16.794998    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 09 03:44:17.794327 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:17.794293    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 09 03:44:17.794773 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:17.794346    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 09 03:44:18.794364 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:18.794327    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 09 03:44:18.794763 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:18.794388    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 09 03:44:19.794863 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:19.794832    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 09 03:44:19.795248 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:19.794883    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 09 03:44:20.794512 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:20.794475    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 09 03:44:20.794710 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:20.794537    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 09 03:44:21.798543 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:21.798510    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:21.798970 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:21.798950    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:21.799767 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:21.799751    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:34.419361 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:34.419280    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"]
Sep 09 03:44:34.419844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:34.419576    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" containerID="cri-o://a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015" gracePeriod=30
Sep 09 03:44:36.801023 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:36.800987    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 03:44:36.801402 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:36.801055    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:44:39.670609 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.670586    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:39.717509 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717478    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-istio-token\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90abf08f-f646-4529-98d4-6b3d019a3454-istio-podinfo\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717576    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm4zj\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-kube-api-access-qm4zj\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717616    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-socket\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90abf08f-f646-4529-98d4-6b3d019a3454-istiod-ca-cert\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-credential-socket\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717793    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-certs\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.717870 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-envoy\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.718062 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-data\") pod \"90abf08f-f646-4529-98d4-6b3d019a3454\" (UID: \"90abf08f-f646-4529-98d4-6b3d019a3454\") "
Sep 09 03:44:39.718062 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.717977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:44:39.718211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.718187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:44:39.718276 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.718248    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.718399 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.718380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-data" (OuterVolumeSpecName: "istio-data") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:44:39.718567 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.718540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:44:39.718754 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.718732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90abf08f-f646-4529-98d4-6b3d019a3454-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:44:39.720227 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.720202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90abf08f-f646-4529-98d4-6b3d019a3454-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:44:39.720547 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.720516    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-istio-token" (OuterVolumeSpecName: "istio-token") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:44:39.720547 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.720534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:44:39.720711 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.720556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-kube-api-access-qm4zj" (OuterVolumeSpecName: "kube-api-access-qm4zj") pod "90abf08f-f646-4529-98d4-6b3d019a3454" (UID: "90abf08f-f646-4529-98d4-6b3d019a3454"). InnerVolumeSpecName "kube-api-access-qm4zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:44:39.819079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819056    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90abf08f-f646-4529-98d4-6b3d019a3454-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819079 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819079    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819088    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819098    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819106    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90abf08f-f646-4529-98d4-6b3d019a3454-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819115    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819122    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90abf08f-f646-4529-98d4-6b3d019a3454-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:39.819250 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:39.819130    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm4zj\" (UniqueName: \"kubernetes.io/projected/90abf08f-f646-4529-98d4-6b3d019a3454-kube-api-access-qm4zj\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:44:40.180903 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.180819    2580 generic.go:356] "Generic (PLEG): container finished" podID="90abf08f-f646-4529-98d4-6b3d019a3454" containerID="a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015" exitCode=0
Sep 09 03:44:40.180903 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.180886    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"
Sep 09 03:44:40.181082 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.180913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" event={"ID":"90abf08f-f646-4529-98d4-6b3d019a3454","Type":"ContainerDied","Data":"a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015"}
Sep 09 03:44:40.181082 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.181000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8" event={"ID":"90abf08f-f646-4529-98d4-6b3d019a3454","Type":"ContainerDied","Data":"615a185e6f9e3a21fc5976fc3e6a35c423c71a8d90af81036c9e3bff9a916641"}
Sep 09 03:44:40.181082 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.181028    2580 scope.go:117] "RemoveContainer" containerID="a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015"
Sep 09 03:44:40.191477 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.191458    2580 scope.go:117] "RemoveContainer" containerID="48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"
Sep 09 03:44:40.201521 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.201488    2580 scope.go:117] "RemoveContainer" containerID="a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015"
Sep 09 03:44:40.201942 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:44:40.201913    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015\": container with ID starting with a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015 not found: ID does not exist" containerID="a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015"
Sep 09 03:44:40.202111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.201957    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015"} err="failed to get container status \"a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015\": rpc error: code = NotFound desc = could not find container \"a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015\": container with ID starting with a01b62a4407161ca1f2939b11600879d9d75cc6873d3313e98e1f8e511d11015 not found: ID does not exist"
Sep 09 03:44:40.202111 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.201980    2580 scope.go:117] "RemoveContainer" containerID="48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"
Sep 09 03:44:40.202489 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:44:40.202466    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f\": container with ID starting with 48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f not found: ID does not exist" containerID="48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"
Sep 09 03:44:40.202489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.202486    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"]
Sep 09 03:44:40.202613 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.202495    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f"} err="failed to get container status \"48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f\": rpc error: code = NotFound desc = could not find container \"48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f\": container with ID starting with 48733c1d91b30cb13fcfb65e77e1cf255d262aa7a2c0aac9bcfb029f0d2aa52f not found: ID does not exist"
Sep 09 03:44:40.205269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:40.205251    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-65a459-openshift-default-8c87fb664-swmq8"]
Sep 09 03:44:41.790247 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:44:41.790218    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" path="/var/lib/kubelet/pods/90abf08f-f646-4529-98d4-6b3d019a3454/volumes"
Sep 09 03:45:00.144706 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.144655    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"]
Sep 09 03:45:00.145209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.145107    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy"
Sep 09 03:45:00.145209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.145119    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy"
Sep 09 03:45:00.145209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.145127    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy"
Sep 09 03:45:00.145209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.145133    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy"
Sep 09 03:45:00.145209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.145202    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy"
Sep 09 03:45:00.145209 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.145211    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90abf08f-f646-4529-98d4-6b3d019a3454" containerName="istio-proxy"
Sep 09 03:45:00.148569 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.148552    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"]
Sep 09 03:45:00.148697 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.148661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"
Sep 09 03:45:00.151685 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.151650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w7q9f\""
Sep 09 03:45:00.152177 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.152159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"
Sep 09 03:45:00.161427 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.161405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"]
Sep 09 03:45:00.164199 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.164179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"
Sep 09 03:45:00.164477 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.164454    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"]
Sep 09 03:45:00.173001 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.172986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"
Sep 09 03:45:00.297391 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.297361    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"]
Sep 09 03:45:00.446875 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.446845    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"]
Sep 09 03:45:00.448308 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:45:00.448282    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbe7e91_7252_46a9_ba2a_627c0efa096b.slice/crio-489250e6569ddb83b5edb8469efbbb123f32d26ff660718eb2dbf353235bde93 WatchSource:0}: Error finding container 489250e6569ddb83b5edb8469efbbb123f32d26ff660718eb2dbf353235bde93: Status 404 returned error can't find the container with id 489250e6569ddb83b5edb8469efbbb123f32d26ff660718eb2dbf353235bde93
Sep 09 03:45:00.464946 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:00.464923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"]
Sep 09 03:45:00.465442 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:45:00.465411    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427a0780_7a8e_4920_8714_f24301264c62.slice/crio-09a590d39a570a9dc5b1960cf4e67a9976d3260b10ecffe0ec72175690237308 WatchSource:0}: Error finding container 09a590d39a570a9dc5b1960cf4e67a9976d3260b10ecffe0ec72175690237308: Status 404 returned error can't find the container with id 09a590d39a570a9dc5b1960cf4e67a9976d3260b10ecffe0ec72175690237308
Sep 09 03:45:01.315254 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:01.315212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7" event={"ID":"427a0780-7a8e-4920-8714-f24301264c62","Type":"ContainerStarted","Data":"dceef7ee62670d15b25d9382d5a343b6e760dc472def4f8c9d0aa7880f6075f5"}
Sep 09 03:45:01.315745 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:01.315275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7" event={"ID":"427a0780-7a8e-4920-8714-f24301264c62","Type":"ContainerStarted","Data":"09a590d39a570a9dc5b1960cf4e67a9976d3260b10ecffe0ec72175690237308"}
Sep 09 03:45:01.316798 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:01.316772    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm" event={"ID":"5cbe7e91-7252-46a9-ba2a-627c0efa096b","Type":"ContainerStarted","Data":"916cc55739b903d748cbb2439f08b096a60553453286589e60b89e61315f1c86"}
Sep 09 03:45:01.316883 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:01.316824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm" event={"ID":"5cbe7e91-7252-46a9-ba2a-627c0efa096b","Type":"ContainerStarted","Data":"489250e6569ddb83b5edb8469efbbb123f32d26ff660718eb2dbf353235bde93"}
Sep 09 03:45:01.331023 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:01.330967    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7" podStartSLOduration=1.330949774 podStartE2EDuration="1.330949774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:45:01.329751715 +0000 UTC m=+1912.029515677" watchObservedRunningTime="2026-09-09 03:45:01.330949774 +0000 UTC m=+1912.030713739"
Sep 09 03:45:01.346165 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:01.346114    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm" podStartSLOduration=1.346102837 podStartE2EDuration="1.346102837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:45:01.344577851 +0000 UTC m=+1912.044341814" watchObservedRunningTime="2026-09-09 03:45:01.346102837 +0000 UTC m=+1912.045866799"
Sep 09 03:45:02.324689 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:02.324637    2580 generic.go:356] "Generic (PLEG): container finished" podID="5cbe7e91-7252-46a9-ba2a-627c0efa096b" containerID="916cc55739b903d748cbb2439f08b096a60553453286589e60b89e61315f1c86" exitCode=0
Sep 09 03:45:02.325087 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:02.324726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm" event={"ID":"5cbe7e91-7252-46a9-ba2a-627c0efa096b","Type":"ContainerDied","Data":"916cc55739b903d748cbb2439f08b096a60553453286589e60b89e61315f1c86"}
Sep 09 03:45:02.326585 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:02.326558    2580 generic.go:356] "Generic (PLEG): container finished" podID="427a0780-7a8e-4920-8714-f24301264c62" containerID="dceef7ee62670d15b25d9382d5a343b6e760dc472def4f8c9d0aa7880f6075f5" exitCode=0
Sep 09 03:45:02.326742 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:02.326636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7" event={"ID":"427a0780-7a8e-4920-8714-f24301264c62","Type":"ContainerDied","Data":"dceef7ee62670d15b25d9382d5a343b6e760dc472def4f8c9d0aa7880f6075f5"}
Sep 09 03:45:03.474638 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:03.474613    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"
Sep 09 03:45:03.514844 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:03.514820    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"
Sep 09 03:45:04.350489 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:04.350457    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"
Sep 09 03:45:04.350723 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:04.350494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm" event={"ID":"5cbe7e91-7252-46a9-ba2a-627c0efa096b","Type":"ContainerDied","Data":"489250e6569ddb83b5edb8469efbbb123f32d26ff660718eb2dbf353235bde93"}
Sep 09 03:45:04.350723 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:04.350620    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="489250e6569ddb83b5edb8469efbbb123f32d26ff660718eb2dbf353235bde93"
Sep 09 03:45:04.352145 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:04.352127    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7"
Sep 09 03:45:04.352269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:04.352176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815425-tttr7" event={"ID":"427a0780-7a8e-4920-8714-f24301264c62","Type":"ContainerDied","Data":"09a590d39a570a9dc5b1960cf4e67a9976d3260b10ecffe0ec72175690237308"}
Sep 09 03:45:04.352269 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:04.352223    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09a590d39a570a9dc5b1960cf4e67a9976d3260b10ecffe0ec72175690237308"
Sep 09 03:45:15.325634 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.325565    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk" podUID="0463ff49-c9fb-470f-a7c8-41c200173422" containerName="main" containerID="cri-o://18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0" gracePeriod=45
Sep 09 03:45:15.578080 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.578021    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:45:15.746254 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-tmp-dir\") pod \"0463ff49-c9fb-470f-a7c8-41c200173422\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") "
Sep 09 03:45:15.746418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-model-cache\") pod \"0463ff49-c9fb-470f-a7c8-41c200173422\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") "
Sep 09 03:45:15.746418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746315    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-dshm\") pod \"0463ff49-c9fb-470f-a7c8-41c200173422\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") "
Sep 09 03:45:15.746418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746336    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcmtz\" (UniqueName: \"kubernetes.io/projected/0463ff49-c9fb-470f-a7c8-41c200173422-kube-api-access-vcmtz\") pod \"0463ff49-c9fb-470f-a7c8-41c200173422\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") "
Sep 09 03:45:15.746418 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0463ff49-c9fb-470f-a7c8-41c200173422-tls-certs\") pod \"0463ff49-c9fb-470f-a7c8-41c200173422\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") "
Sep 09 03:45:15.746631 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746452    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-home\") pod \"0463ff49-c9fb-470f-a7c8-41c200173422\" (UID: \"0463ff49-c9fb-470f-a7c8-41c200173422\") "
Sep 09 03:45:15.746631 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0463ff49-c9fb-470f-a7c8-41c200173422" (UID: "0463ff49-c9fb-470f-a7c8-41c200173422"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:15.746631 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746573    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-model-cache" (OuterVolumeSpecName: "model-cache") pod "0463ff49-c9fb-470f-a7c8-41c200173422" (UID: "0463ff49-c9fb-470f-a7c8-41c200173422"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:15.746832 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746749    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-home" (OuterVolumeSpecName: "home") pod "0463ff49-c9fb-470f-a7c8-41c200173422" (UID: "0463ff49-c9fb-470f-a7c8-41c200173422"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:15.746915 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746898    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:15.746966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746917    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:15.746966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.746926    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:15.748808 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.748784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0463ff49-c9fb-470f-a7c8-41c200173422-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0463ff49-c9fb-470f-a7c8-41c200173422" (UID: "0463ff49-c9fb-470f-a7c8-41c200173422"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:45:15.749089 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.749062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-dshm" (OuterVolumeSpecName: "dshm") pod "0463ff49-c9fb-470f-a7c8-41c200173422" (UID: "0463ff49-c9fb-470f-a7c8-41c200173422"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:15.749183 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.749085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0463ff49-c9fb-470f-a7c8-41c200173422-kube-api-access-vcmtz" (OuterVolumeSpecName: "kube-api-access-vcmtz") pod "0463ff49-c9fb-470f-a7c8-41c200173422" (UID: "0463ff49-c9fb-470f-a7c8-41c200173422"). InnerVolumeSpecName "kube-api-access-vcmtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:45:15.847920 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.847867    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0463ff49-c9fb-470f-a7c8-41c200173422-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:15.847920 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.847887    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcmtz\" (UniqueName: \"kubernetes.io/projected/0463ff49-c9fb-470f-a7c8-41c200173422-kube-api-access-vcmtz\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:15.847920 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:15.847896    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0463ff49-c9fb-470f-a7c8-41c200173422-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:16.432136 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.432103    2580 generic.go:356] "Generic (PLEG): container finished" podID="0463ff49-c9fb-470f-a7c8-41c200173422" containerID="18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0" exitCode=0
Sep 09 03:45:16.432591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.432149    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"
Sep 09 03:45:16.432591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.432173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk" event={"ID":"0463ff49-c9fb-470f-a7c8-41c200173422","Type":"ContainerDied","Data":"18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0"}
Sep 09 03:45:16.432591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.432203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk" event={"ID":"0463ff49-c9fb-470f-a7c8-41c200173422","Type":"ContainerDied","Data":"354c33197f42696846d3141d3b272bb335673d4e20ebe769bd0fa4ac7fd8d0a7"}
Sep 09 03:45:16.432591 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.432219    2580 scope.go:117] "RemoveContainer" containerID="18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0"
Sep 09 03:45:16.442144 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.442127    2580 scope.go:117] "RemoveContainer" containerID="18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0"
Sep 09 03:45:16.442412 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:45:16.442393    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0\": container with ID starting with 18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0 not found: ID does not exist" containerID="18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0"
Sep 09 03:45:16.442458 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.442421    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0"} err="failed to get container status \"18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0\": rpc error: code = NotFound desc = could not find container \"18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0\": container with ID starting with 18a13d750a7a8db4f93226389464978bc485e2806826e4efe172e0666fa1f9c0 not found: ID does not exist"
Sep 09 03:45:16.450723 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.450656    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"]
Sep 09 03:45:16.454095 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:16.454076    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-46be70/ipp-route-46be70-kserve-57df4c8587-7rzdk"]
Sep 09 03:45:17.790076 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:17.790036    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0463ff49-c9fb-470f-a7c8-41c200173422" path="/var/lib/kubelet/pods/0463ff49-c9fb-470f-a7c8-41c200173422/volumes"
Sep 09 03:45:43.694658 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:43.694625    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"]
Sep 09 03:45:43.695231 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:43.694914    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy" containerID="cri-o://4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf" gracePeriod=30
Sep 09 03:45:48.954949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:48.954923    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:45:49.037662 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037628    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-envoy\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.037842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-token\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.037842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-certs\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.037842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-credential-socket\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.037842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037805    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d786e82e-32e5-4792-9fdb-58eda90cd466-istiod-ca-cert\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.037842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037836    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-data\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.038126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037866    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-podinfo\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.038126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d94f\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-kube-api-access-7d94f\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.038126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.037985    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-socket\") pod \"d786e82e-32e5-4792-9fdb-58eda90cd466\" (UID: \"d786e82e-32e5-4792-9fdb-58eda90cd466\") "
Sep 09 03:45:49.038126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:49.038126 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038058    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:49.038595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038143    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-data" (OuterVolumeSpecName: "istio-data") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:49.038595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038281    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:49.038595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038452    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d786e82e-32e5-4792-9fdb-58eda90cd466-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:45:49.038802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038558    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.038802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038625    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.038802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038640    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.038802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.038654    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.040114 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.040086    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:45:49.040249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.040122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-token" (OuterVolumeSpecName: "istio-token") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:45:49.040249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.040151    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:45:49.040249 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.040171    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-kube-api-access-7d94f" (OuterVolumeSpecName: "kube-api-access-7d94f") pod "d786e82e-32e5-4792-9fdb-58eda90cd466" (UID: "d786e82e-32e5-4792-9fdb-58eda90cd466"). InnerVolumeSpecName "kube-api-access-7d94f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:45:49.140040 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.140020    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.140040 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.140037    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.140214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.140046    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d786e82e-32e5-4792-9fdb-58eda90cd466-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.140214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.140054    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d786e82e-32e5-4792-9fdb-58eda90cd466-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.140214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.140061    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d94f\" (UniqueName: \"kubernetes.io/projected/d786e82e-32e5-4792-9fdb-58eda90cd466-kube-api-access-7d94f\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:45:49.646073 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.646040    2580 generic.go:356] "Generic (PLEG): container finished" podID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerID="4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf" exitCode=0
Sep 09 03:45:49.646271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.646077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" event={"ID":"d786e82e-32e5-4792-9fdb-58eda90cd466","Type":"ContainerDied","Data":"4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf"}
Sep 09 03:45:49.646271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.646084    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"
Sep 09 03:45:49.646271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.646130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv" event={"ID":"d786e82e-32e5-4792-9fdb-58eda90cd466","Type":"ContainerDied","Data":"58f178ad4340b430d1d4a7d0b8d86dd3461fdeba6ef073d93bccc6df31563ebb"}
Sep 09 03:45:49.646271 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.646158    2580 scope.go:117] "RemoveContainer" containerID="4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf"
Sep 09 03:45:49.657013 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.656996    2580 scope.go:117] "RemoveContainer" containerID="a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"
Sep 09 03:45:49.666933 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.666696    2580 scope.go:117] "RemoveContainer" containerID="4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf"
Sep 09 03:45:49.667292 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:45:49.667255    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf\": container with ID starting with 4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf not found: ID does not exist" containerID="4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf"
Sep 09 03:45:49.667380 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.667291    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf"} err="failed to get container status \"4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf\": rpc error: code = NotFound desc = could not find container \"4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf\": container with ID starting with 4cc9c559362d19c35271ef01d3384f8083a27d8c18c2335030c8e901b40784cf not found: ID does not exist"
Sep 09 03:45:49.667380 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.667313    2580 scope.go:117] "RemoveContainer" containerID="a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"
Sep 09 03:45:49.667582 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:45:49.667563    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53\": container with ID starting with a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53 not found: ID does not exist" containerID="a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"
Sep 09 03:45:49.667624 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.667590    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53"} err="failed to get container status \"a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53\": rpc error: code = NotFound desc = could not find container \"a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53\": container with ID starting with a9e5098e84ac6cf38826eadca5f4f4c7b403b08d5ecb5ad99b25bde8e9591b53 not found: ID does not exist"
Sep 09 03:45:49.669752 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.669731    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"]
Sep 09 03:45:49.673537 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.673516    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-46be70-openshift-default-848677f594-fq4xv"]
Sep 09 03:45:49.796330 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:45:49.796299    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" path="/var/lib/kubelet/pods/d786e82e-32e5-4792-9fdb-58eda90cd466/volumes"
Sep 09 03:46:01.459213 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:01.459129    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"]
Sep 09 03:46:01.459584 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:01.459418    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy" containerID="cri-o://2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083" gracePeriod=30
Sep 09 03:46:06.719446 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.719423    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:46:06.757410 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.757326    2580 generic.go:356] "Generic (PLEG): container finished" podID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerID="2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083" exitCode=0
Sep 09 03:46:06.757410 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.757380    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"
Sep 09 03:46:06.757410 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.757398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" event={"ID":"064a51ff-d3fc-4feb-928a-f9cce6e2c437","Type":"ContainerDied","Data":"2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083"}
Sep 09 03:46:06.757700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.757483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6" event={"ID":"064a51ff-d3fc-4feb-928a-f9cce6e2c437","Type":"ContainerDied","Data":"ef7a856aac2f584474a0393b8e15a5f273e5a8332988dd02dfe8b04ccc00a01d"}
Sep 09 03:46:06.757700 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.757509    2580 scope.go:117] "RemoveContainer" containerID="2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083"
Sep 09 03:46:06.767205 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.767183    2580 scope.go:117] "RemoveContainer" containerID="d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"
Sep 09 03:46:06.775713 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.775694    2580 scope.go:117] "RemoveContainer" containerID="2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083"
Sep 09 03:46:06.775963 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:46:06.775944    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083\": container with ID starting with 2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083 not found: ID does not exist" containerID="2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083"
Sep 09 03:46:06.776025 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.775975    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083"} err="failed to get container status \"2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083\": rpc error: code = NotFound desc = could not find container \"2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083\": container with ID starting with 2270b91ed574996bc72b6a7e6f07debe80b3f808d55ee68325380b632a255083 not found: ID does not exist"
Sep 09 03:46:06.776025 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.775992    2580 scope.go:117] "RemoveContainer" containerID="d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"
Sep 09 03:46:06.776230 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:46:06.776214    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f\": container with ID starting with d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f not found: ID does not exist" containerID="d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"
Sep 09 03:46:06.776278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.776235    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f"} err="failed to get container status \"d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f\": rpc error: code = NotFound desc = could not find container \"d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f\": container with ID starting with d7b2d26916ed9688ae179eefbccea34c5be65fc6279654edb3fc938fdd07628f not found: ID does not exist"
Sep 09 03:46:06.791206 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-token\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791282 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791228    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istiod-ca-cert\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791282 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791247    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-credential-socket\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791379 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-certs\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791436 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-data\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791436 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zstrg\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-kube-api-access-zstrg\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791436 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791431    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-podinfo\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791585 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-envoy\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791585 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-socket\") pod \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\" (UID: \"064a51ff-d3fc-4feb-928a-f9cce6e2c437\") "
Sep 09 03:46:06.791585 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791565    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:06.791765 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791595    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:06.791765 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791619    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:06.791765 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-data" (OuterVolumeSpecName: "istio-data") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:06.791954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791889    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.791954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791914    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.791954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791928    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.791954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791942    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.792159 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.791949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:06.793408 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.793381    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-token" (OuterVolumeSpecName: "istio-token") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:06.793747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.793720    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:46:06.793861 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.793843    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:06.793954 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.793935    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-kube-api-access-zstrg" (OuterVolumeSpecName: "kube-api-access-zstrg") pod "064a51ff-d3fc-4feb-928a-f9cce6e2c437" (UID: "064a51ff-d3fc-4feb-928a-f9cce6e2c437"). InnerVolumeSpecName "kube-api-access-zstrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:06.892438 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.892415    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zstrg\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-kube-api-access-zstrg\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.892438 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.892438    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.892556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.892447    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.892556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.892456    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/064a51ff-d3fc-4feb-928a-f9cce6e2c437-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:06.892556 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:06.892464    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/064a51ff-d3fc-4feb-928a-f9cce6e2c437-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:07.080797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:07.080770    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"]
Sep 09 03:46:07.085595 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:07.085571    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-60a731-openshift-default-84b7fb6868-lg6c6"]
Sep 09 03:46:07.789532 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:07.789502    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" path="/var/lib/kubelet/pods/064a51ff-d3fc-4feb-928a-f9cce6e2c437/volumes"
Sep 09 03:46:10.123312 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:10.123283    2580 scope.go:117] "RemoveContainer" containerID="d196c1dd99bb0bb16dde59de38bd917d95edb77cd36b1b38c15ce56f95da26bb"
Sep 09 03:46:12.482794 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:12.482763    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"]
Sep 09 03:46:12.485535 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:12.485507    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d8e85-29815425-kntgm"]
Sep 09 03:46:13.789773 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:13.789737    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cbe7e91-7252-46a9-ba2a-627c0efa096b" path="/var/lib/kubelet/pods/5cbe7e91-7252-46a9-ba2a-627c0efa096b/volumes"
Sep 09 03:46:19.381009 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:19.380980    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"]
Sep 09 03:46:19.381448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:19.381220    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy" containerID="cri-o://4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39" gracePeriod=30
Sep 09 03:46:24.639999 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.639973    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:46:24.740791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.740761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c88bm\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-kube-api-access-c88bm\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.740949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.740799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-envoy\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.740949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.740885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-certs\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.740949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.740914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf0e463-819e-4776-9224-8b05787dd61c-istiod-ca-cert\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.740949 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.740946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf0e463-819e-4776-9224-8b05787dd61c-istio-podinfo\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.741174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.740968    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-socket\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.741174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741009    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-data\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.741174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741036    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-istio-token\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.741174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741090    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-credential-socket\") pod \"1cf0e463-819e-4776-9224-8b05787dd61c\" (UID: \"1cf0e463-819e-4776-9224-8b05787dd61c\") "
Sep 09 03:46:24.741174 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741127    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:24.741420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741282    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:24.741420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741328    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.741420 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741323    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf0e463-819e-4776-9224-8b05787dd61c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:46:24.741622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741598    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:24.741706 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.741518    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:24.743160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.743127    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-kube-api-access-c88bm" (OuterVolumeSpecName: "kube-api-access-c88bm") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "kube-api-access-c88bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:24.743160 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.743146    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:46:24.743353 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.743336    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:46:24.743480 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.743456    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cf0e463-819e-4776-9224-8b05787dd61c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cf0e463-819e-4776-9224-8b05787dd61c" (UID: "1cf0e463-819e-4776-9224-8b05787dd61c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 03:46:24.842538 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842498    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cf0e463-819e-4776-9224-8b05787dd61c-istiod-ca-cert\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842538 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842534    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cf0e463-819e-4776-9224-8b05787dd61c-istio-podinfo\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842538 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842543    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-workload-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842538 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842553    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-data\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842806 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842561    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-istio-token\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842806 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842569    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-credential-socket\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842806 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842578    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c88bm\" (UniqueName: \"kubernetes.io/projected/1cf0e463-819e-4776-9224-8b05787dd61c-kube-api-access-c88bm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.842806 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.842586    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cf0e463-819e-4776-9224-8b05787dd61c-istio-envoy\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:46:24.866339 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.866316    2580 generic.go:356] "Generic (PLEG): container finished" podID="1cf0e463-819e-4776-9224-8b05787dd61c" containerID="4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39" exitCode=0
Sep 09 03:46:24.866449 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.866362    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"
Sep 09 03:46:24.866449 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.866400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" event={"ID":"1cf0e463-819e-4776-9224-8b05787dd61c","Type":"ContainerDied","Data":"4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39"}
Sep 09 03:46:24.866537 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.866457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h" event={"ID":"1cf0e463-819e-4776-9224-8b05787dd61c","Type":"ContainerDied","Data":"99efb4f01c57712c2d2c02ac912220a7d136fc37d0c712abff1e0d7b55a13123"}
Sep 09 03:46:24.866537 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.866487    2580 scope.go:117] "RemoveContainer" containerID="4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39"
Sep 09 03:46:24.875977 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.875962    2580 scope.go:117] "RemoveContainer" containerID="6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"
Sep 09 03:46:24.886732 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.886703    2580 scope.go:117] "RemoveContainer" containerID="4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39"
Sep 09 03:46:24.887050 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:46:24.887013    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39\": container with ID starting with 4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39 not found: ID does not exist" containerID="4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39"
Sep 09 03:46:24.887246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.887050    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39"} err="failed to get container status \"4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39\": rpc error: code = NotFound desc = could not find container \"4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39\": container with ID starting with 4a3eab9b3018b5b2b217163fc8a0d55ac304189d749872f64dd0e59bdb061d39 not found: ID does not exist"
Sep 09 03:46:24.887246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.887074    2580 scope.go:117] "RemoveContainer" containerID="6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"
Sep 09 03:46:24.887513 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:46:24.887491    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1\": container with ID starting with 6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1 not found: ID does not exist" containerID="6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"
Sep 09 03:46:24.887587 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.887520    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1"} err="failed to get container status \"6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1\": rpc error: code = NotFound desc = could not find container \"6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1\": container with ID starting with 6da63ff2edcd78c40065a3100dd3311ad48455cf85d71f6a6c16d68fe21137f1 not found: ID does not exist"
Sep 09 03:46:24.889995 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.889977    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"]
Sep 09 03:46:24.895301 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:24.895280    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-5d8e85-openshift-default-6d6bc56ddb-tfn2h"]
Sep 09 03:46:25.789641 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:46:25.789605    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" path="/var/lib/kubelet/pods/1cf0e463-819e-4776-9224-8b05787dd61c/volumes"
Sep 09 03:49:10.810720 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:10.810616    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"]
Sep 09 03:49:10.811289 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:10.810882    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9" podUID="2f504846-7c37-4b8c-af06-2c234fd80f48" containerName="manager" containerID="cri-o://0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4" gracePeriod=10
Sep 09 03:49:13.269998 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.269975    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:49:13.411925 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.411845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f504846-7c37-4b8c-af06-2c234fd80f48-extensions-socket-volume\") pod \"2f504846-7c37-4b8c-af06-2c234fd80f48\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") "
Sep 09 03:49:13.412075 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.411940    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l97hq\" (UniqueName: \"kubernetes.io/projected/2f504846-7c37-4b8c-af06-2c234fd80f48-kube-api-access-l97hq\") pod \"2f504846-7c37-4b8c-af06-2c234fd80f48\" (UID: \"2f504846-7c37-4b8c-af06-2c234fd80f48\") "
Sep 09 03:49:13.412213 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.412187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f504846-7c37-4b8c-af06-2c234fd80f48-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2f504846-7c37-4b8c-af06-2c234fd80f48" (UID: "2f504846-7c37-4b8c-af06-2c234fd80f48"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:49:13.414097 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.414074    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f504846-7c37-4b8c-af06-2c234fd80f48-kube-api-access-l97hq" (OuterVolumeSpecName: "kube-api-access-l97hq") pod "2f504846-7c37-4b8c-af06-2c234fd80f48" (UID: "2f504846-7c37-4b8c-af06-2c234fd80f48"). InnerVolumeSpecName "kube-api-access-l97hq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:49:13.513032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.513009    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l97hq\" (UniqueName: \"kubernetes.io/projected/2f504846-7c37-4b8c-af06-2c234fd80f48-kube-api-access-l97hq\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:49:13.513032 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.513030    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f504846-7c37-4b8c-af06-2c234fd80f48-extensions-socket-volume\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:49:13.856644 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.856612    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f504846-7c37-4b8c-af06-2c234fd80f48" containerID="0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4" exitCode=0
Sep 09 03:49:13.856802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.856688    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"
Sep 09 03:49:13.856802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.856701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9" event={"ID":"2f504846-7c37-4b8c-af06-2c234fd80f48","Type":"ContainerDied","Data":"0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4"}
Sep 09 03:49:13.856802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.856765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9" event={"ID":"2f504846-7c37-4b8c-af06-2c234fd80f48","Type":"ContainerDied","Data":"718b1147364ed49eec80c5478e0efb76ab8c650562cae962e6d81577dd27bc42"}
Sep 09 03:49:13.856802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.856793    2580 scope.go:117] "RemoveContainer" containerID="0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4"
Sep 09 03:49:13.866838 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.866821    2580 scope.go:117] "RemoveContainer" containerID="0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4"
Sep 09 03:49:13.867113 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:49:13.867093    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4\": container with ID starting with 0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4 not found: ID does not exist" containerID="0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4"
Sep 09 03:49:13.867179 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.867124    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4"} err="failed to get container status \"0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4\": rpc error: code = NotFound desc = could not find container \"0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4\": container with ID starting with 0ea0cb9d5575593ea17eeb720e3187ac8ec710cf681eb04b9fa4d7f831660da4 not found: ID does not exist"
Sep 09 03:49:13.875996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.875970    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"]
Sep 09 03:49:13.880167 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:13.880146    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nx6v9"]
Sep 09 03:49:15.789020 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:49:15.788991    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f504846-7c37-4b8c-af06-2c234fd80f48" path="/var/lib/kubelet/pods/2f504846-7c37-4b8c-af06-2c234fd80f48/volumes"
Sep 09 03:50:11.547211 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547176    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"]
Sep 09 03:50:11.547608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547573    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy"
Sep 09 03:50:11.547608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547585    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy"
Sep 09 03:50:11.547608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547603    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0463ff49-c9fb-470f-a7c8-41c200173422" containerName="main"
Sep 09 03:50:11.547608 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547609    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0463ff49-c9fb-470f-a7c8-41c200173422" containerName="main"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547620    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547626    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547633    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="427a0780-7a8e-4920-8714-f24301264c62" containerName="cleanup"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547639    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="427a0780-7a8e-4920-8714-f24301264c62" containerName="cleanup"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547646    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547652    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547659    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547664    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547694    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547700    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547705    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbe7e91-7252-46a9-ba2a-627c0efa096b" containerName="cleanup"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547710    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbe7e91-7252-46a9-ba2a-627c0efa096b" containerName="cleanup"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547721    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f504846-7c37-4b8c-af06-2c234fd80f48" containerName="manager"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547725    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f504846-7c37-4b8c-af06-2c234fd80f48" containerName="manager"
Sep 09 03:50:11.547788 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547795    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f504846-7c37-4b8c-af06-2c234fd80f48" containerName="manager"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547804    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547811    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547818    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547823    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="064a51ff-d3fc-4feb-928a-f9cce6e2c437" containerName="istio-proxy"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547829    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0463ff49-c9fb-470f-a7c8-41c200173422" containerName="main"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547835    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cbe7e91-7252-46a9-ba2a-627c0efa096b" containerName="cleanup"
Sep 09 03:50:11.548236 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.547845    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="427a0780-7a8e-4920-8714-f24301264c62" containerName="cleanup"
Sep 09 03:50:11.552130 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.552113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.554770 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.554751    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gdnld\""
Sep 09 03:50:11.564237 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.564214    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"]
Sep 09 03:50:11.666578 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.666552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a106ada-be9e-4e83-9cad-bd148837b33d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nrqvk\" (UID: \"8a106ada-be9e-4e83-9cad-bd148837b33d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.666728 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.666598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75rwf\" (UniqueName: \"kubernetes.io/projected/8a106ada-be9e-4e83-9cad-bd148837b33d-kube-api-access-75rwf\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nrqvk\" (UID: \"8a106ada-be9e-4e83-9cad-bd148837b33d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.767956 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.767931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a106ada-be9e-4e83-9cad-bd148837b33d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nrqvk\" (UID: \"8a106ada-be9e-4e83-9cad-bd148837b33d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.768048 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.767979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75rwf\" (UniqueName: \"kubernetes.io/projected/8a106ada-be9e-4e83-9cad-bd148837b33d-kube-api-access-75rwf\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nrqvk\" (UID: \"8a106ada-be9e-4e83-9cad-bd148837b33d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.768257 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.768241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a106ada-be9e-4e83-9cad-bd148837b33d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nrqvk\" (UID: \"8a106ada-be9e-4e83-9cad-bd148837b33d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.780454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.780434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75rwf\" (UniqueName: \"kubernetes.io/projected/8a106ada-be9e-4e83-9cad-bd148837b33d-kube-api-access-75rwf\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nrqvk\" (UID: \"8a106ada-be9e-4e83-9cad-bd148837b33d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:11.863241 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:11.863183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:12.144966 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:12.144935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"]
Sep 09 03:50:12.145228 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:50:12.145202    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a106ada_be9e_4e83_9cad_bd148837b33d.slice/crio-3208b6881990a8bd62d992df9156f401481e9fe97e9daf7b8cd79dec393050a5 WatchSource:0}: Error finding container 3208b6881990a8bd62d992df9156f401481e9fe97e9daf7b8cd79dec393050a5: Status 404 returned error can't find the container with id 3208b6881990a8bd62d992df9156f401481e9fe97e9daf7b8cd79dec393050a5
Sep 09 03:50:12.193697 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:12.193657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk" event={"ID":"8a106ada-be9e-4e83-9cad-bd148837b33d","Type":"ContainerStarted","Data":"3208b6881990a8bd62d992df9156f401481e9fe97e9daf7b8cd79dec393050a5"}
Sep 09 03:50:13.201864 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:13.201834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk" event={"ID":"8a106ada-be9e-4e83-9cad-bd148837b33d","Type":"ContainerStarted","Data":"9216bcc099a27b3541e43e6d7676caa3d3138716cb9bf524505f8fc93597a887"}
Sep 09 03:50:13.202251 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:13.201964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:50:13.225968 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:13.225917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk" podStartSLOduration=2.225904644 podStartE2EDuration="2.225904644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:50:13.224781302 +0000 UTC m=+2223.924545266" watchObservedRunningTime="2026-09-09 03:50:13.225904644 +0000 UTC m=+2223.925668606"
Sep 09 03:50:24.210229 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:50:24.210195    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nrqvk"
Sep 09 03:51:05.920200 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.920165    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"]
Sep 09 03:51:05.920802 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.920783    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy"
Sep 09 03:51:05.920895 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.920805    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy"
Sep 09 03:51:05.920955 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.920913    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d786e82e-32e5-4792-9fdb-58eda90cd466" containerName="istio-proxy"
Sep 09 03:51:05.920955 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.920929    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf0e463-819e-4776-9224-8b05787dd61c" containerName="istio-proxy"
Sep 09 03:51:05.924187 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.924168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:05.927163 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.927140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-c5827fc9-kserve-self-signed-certs\""
Sep 09 03:51:05.936067 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:05.936046    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"]
Sep 09 03:51:06.008604 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.008579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-tmp-dir\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.008797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.008638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cxrs\" (UniqueName: \"kubernetes.io/projected/7be1831d-75bf-4106-9f05-d2c98fee0a10-kube-api-access-6cxrs\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.008797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.008664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be1831d-75bf-4106-9f05-d2c98fee0a10-tls-certs\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.008797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.008723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-dshm\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.008797 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.008790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-home\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.008944 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.008805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-model-cache\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110192 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cxrs\" (UniqueName: \"kubernetes.io/projected/7be1831d-75bf-4106-9f05-d2c98fee0a10-kube-api-access-6cxrs\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be1831d-75bf-4106-9f05-d2c98fee0a10-tls-certs\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-dshm\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-home\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110342 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-model-cache\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110554 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-tmp-dir\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110828 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-tmp-dir\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110930 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-home\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.110930 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.110862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-model-cache\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.112819 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.112800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-dshm\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.112980 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.112965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be1831d-75bf-4106-9f05-d2c98fee0a10-tls-certs\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.119704 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.119662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cxrs\" (UniqueName: \"kubernetes.io/projected/7be1831d-75bf-4106-9f05-d2c98fee0a10-kube-api-access-6cxrs\") pod \"e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") " pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.235224 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.235194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:06.511338 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.511312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"]
Sep 09 03:51:06.512273 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:51:06.512245    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7be1831d_75bf_4106_9f05_d2c98fee0a10.slice/crio-4a9e1e35d1d8f1f7ca90e5e8763245efa01a24be4564efcdc65002e4e20ac1e2 WatchSource:0}: Error finding container 4a9e1e35d1d8f1f7ca90e5e8763245efa01a24be4564efcdc65002e4e20ac1e2: Status 404 returned error can't find the container with id 4a9e1e35d1d8f1f7ca90e5e8763245efa01a24be4564efcdc65002e4e20ac1e2
Sep 09 03:51:06.533996 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:06.533964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt" event={"ID":"7be1831d-75bf-4106-9f05-d2c98fee0a10","Type":"ContainerStarted","Data":"4a9e1e35d1d8f1f7ca90e5e8763245efa01a24be4564efcdc65002e4e20ac1e2"}
Sep 09 03:51:07.541460 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:07.541418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt" event={"ID":"7be1831d-75bf-4106-9f05-d2c98fee0a10","Type":"ContainerStarted","Data":"176290f0b5b63ff88fe48e1336ea0689bd068e41ee9bd58543a815daae9ebd43"}
Sep 09 03:51:07.559435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:07.559385    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt" podStartSLOduration=2.559369355 podStartE2EDuration="2.559369355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:51:07.558389175 +0000 UTC m=+2278.258153141" watchObservedRunningTime="2026-09-09 03:51:07.559369355 +0000 UTC m=+2278.259133318"
Sep 09 03:51:10.199505 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:10.199474    2580 scope.go:117] "RemoveContainer" containerID="916cc55739b903d748cbb2439f08b096a60553453286589e60b89e61315f1c86"
Sep 09 03:51:16.235506 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:16.235471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:16.235506 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:16.235517    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:16.247907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:16.247885    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:16.604454 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:16.604377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:51:50.614143 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:51:50.614114    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"]
Sep 09 03:52:05.638125 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:05.638021    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt" podUID="7be1831d-75bf-4106-9f05-d2c98fee0a10" containerName="main" containerID="cri-o://176290f0b5b63ff88fe48e1336ea0689bd068e41ee9bd58543a815daae9ebd43" gracePeriod=45
Sep 09 03:52:05.887769 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:05.887743    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be1831d-75bf-4106-9f05-d2c98fee0a10" containerID="176290f0b5b63ff88fe48e1336ea0689bd068e41ee9bd58543a815daae9ebd43" exitCode=0
Sep 09 03:52:05.887914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:05.887809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt" event={"ID":"7be1831d-75bf-4106-9f05-d2c98fee0a10","Type":"ContainerDied","Data":"176290f0b5b63ff88fe48e1336ea0689bd068e41ee9bd58543a815daae9ebd43"}
Sep 09 03:52:05.887914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:05.887853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt" event={"ID":"7be1831d-75bf-4106-9f05-d2c98fee0a10","Type":"ContainerDied","Data":"4a9e1e35d1d8f1f7ca90e5e8763245efa01a24be4564efcdc65002e4e20ac1e2"}
Sep 09 03:52:05.887914 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:05.887865    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a9e1e35d1d8f1f7ca90e5e8763245efa01a24be4564efcdc65002e4e20ac1e2"
Sep 09 03:52:05.895948 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:05.895899    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:52:06.028486 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-model-cache\") pod \"7be1831d-75bf-4106-9f05-d2c98fee0a10\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") "
Sep 09 03:52:06.028622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-dshm\") pod \"7be1831d-75bf-4106-9f05-d2c98fee0a10\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") "
Sep 09 03:52:06.028622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028550    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-home\") pod \"7be1831d-75bf-4106-9f05-d2c98fee0a10\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") "
Sep 09 03:52:06.028622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be1831d-75bf-4106-9f05-d2c98fee0a10-tls-certs\") pod \"7be1831d-75bf-4106-9f05-d2c98fee0a10\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") "
Sep 09 03:52:06.028622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028601    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cxrs\" (UniqueName: \"kubernetes.io/projected/7be1831d-75bf-4106-9f05-d2c98fee0a10-kube-api-access-6cxrs\") pod \"7be1831d-75bf-4106-9f05-d2c98fee0a10\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") "
Sep 09 03:52:06.028622 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028623    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-tmp-dir\") pod \"7be1831d-75bf-4106-9f05-d2c98fee0a10\" (UID: \"7be1831d-75bf-4106-9f05-d2c98fee0a10\") "
Sep 09 03:52:06.028907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028794    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-model-cache" (OuterVolumeSpecName: "model-cache") pod "7be1831d-75bf-4106-9f05-d2c98fee0a10" (UID: "7be1831d-75bf-4106-9f05-d2c98fee0a10"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:52:06.028907 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-home" (OuterVolumeSpecName: "home") pod "7be1831d-75bf-4106-9f05-d2c98fee0a10" (UID: "7be1831d-75bf-4106-9f05-d2c98fee0a10"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:52:06.029030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028970    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-model-cache\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:06.029030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.028985    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-home\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:06.029030 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.029001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7be1831d-75bf-4106-9f05-d2c98fee0a10" (UID: "7be1831d-75bf-4106-9f05-d2c98fee0a10"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:52:06.030916 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.030892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-dshm" (OuterVolumeSpecName: "dshm") pod "7be1831d-75bf-4106-9f05-d2c98fee0a10" (UID: "7be1831d-75bf-4106-9f05-d2c98fee0a10"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:52:06.031039 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.030964    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7be1831d-75bf-4106-9f05-d2c98fee0a10-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7be1831d-75bf-4106-9f05-d2c98fee0a10" (UID: "7be1831d-75bf-4106-9f05-d2c98fee0a10"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:52:06.031308 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.031282    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be1831d-75bf-4106-9f05-d2c98fee0a10-kube-api-access-6cxrs" (OuterVolumeSpecName: "kube-api-access-6cxrs") pod "7be1831d-75bf-4106-9f05-d2c98fee0a10" (UID: "7be1831d-75bf-4106-9f05-d2c98fee0a10"). InnerVolumeSpecName "kube-api-access-6cxrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:52:06.129421 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.129393    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be1831d-75bf-4106-9f05-d2c98fee0a10-tls-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:06.129524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.129421    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cxrs\" (UniqueName: \"kubernetes.io/projected/7be1831d-75bf-4106-9f05-d2c98fee0a10-kube-api-access-6cxrs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:06.129524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.129437    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-tmp-dir\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:06.129524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.129448    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be1831d-75bf-4106-9f05-d2c98fee0a10-dshm\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:06.894295 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.894261    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"
Sep 09 03:52:06.917300 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.917267    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"]
Sep 09 03:52:06.921860 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:06.921837    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-c5827fc9-kserve-5c95cd99f9-hzpzt"]
Sep 09 03:52:07.789821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:07.789782    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be1831d-75bf-4106-9f05-d2c98fee0a10" path="/var/lib/kubelet/pods/7be1831d-75bf-4106-9f05-d2c98fee0a10/volumes"
Sep 09 03:52:13.147660 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.147624    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"]
Sep 09 03:52:13.148116 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.148051    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be1831d-75bf-4106-9f05-d2c98fee0a10" containerName="main"
Sep 09 03:52:13.148116 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.148063    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be1831d-75bf-4106-9f05-d2c98fee0a10" containerName="main"
Sep 09 03:52:13.148190 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.148125    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be1831d-75bf-4106-9f05-d2c98fee0a10" containerName="main"
Sep 09 03:52:13.151994 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.151975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.154459 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.154444    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-phvzt\""
Sep 09 03:52:13.172624 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.172601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"]
Sep 09 03:52:13.293292 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.293259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-openshift-service-ca\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.293462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.293307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4pdl\" (UniqueName: \"kubernetes.io/projected/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-kube-api-access-g4pdl\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.293462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.293354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-metrics-certs\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.293462 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.293410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-maas-api-tls\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.394316 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.394286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-maas-api-tls\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.394444 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.394341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-openshift-service-ca\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.394444 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.394369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4pdl\" (UniqueName: \"kubernetes.io/projected/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-kube-api-access-g4pdl\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.394444 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.394394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-metrics-certs\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.395042 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.395018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-openshift-service-ca\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.396945 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.396921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-maas-api-tls\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.396945 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.396932    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-metrics-certs\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.403611 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.403556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4pdl\" (UniqueName: \"kubernetes.io/projected/9b2c2b99-cb6c-4513-ae7d-29e7df8df058-kube-api-access-g4pdl\") pod \"maas-api-6578d77458-ld4c8\" (UID: \"9b2c2b99-cb6c-4513-ae7d-29e7df8df058\") " pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.462069 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.462051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:13.743727 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.743703    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"]
Sep 09 03:52:13.744926 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:52:13.744892    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b2c2b99_cb6c_4513_ae7d_29e7df8df058.slice/crio-50b6835530a201c4842a30562dacf0c1adb3dc5d07c939c885bca5c8f1519fe4 WatchSource:0}: Error finding container 50b6835530a201c4842a30562dacf0c1adb3dc5d07c939c885bca5c8f1519fe4: Status 404 returned error can't find the container with id 50b6835530a201c4842a30562dacf0c1adb3dc5d07c939c885bca5c8f1519fe4
Sep 09 03:52:13.746594 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.746573    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:52:13.935814 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:13.935770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8" event={"ID":"9b2c2b99-cb6c-4513-ae7d-29e7df8df058","Type":"ContainerStarted","Data":"50b6835530a201c4842a30562dacf0c1adb3dc5d07c939c885bca5c8f1519fe4"}
Sep 09 03:52:14.943746 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:14.943710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8" event={"ID":"9b2c2b99-cb6c-4513-ae7d-29e7df8df058","Type":"ContainerStarted","Data":"32adc4f144088c38ab51f71b5b3caf47070d9702b6e9f4581f2330fc32874792"}
Sep 09 03:52:14.944196 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:14.943775    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:20.955280 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:20.955252    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8"
Sep 09 03:52:20.978329 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:20.978275    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6578d77458-ld4c8" podStartSLOduration=7.609019639 podStartE2EDuration="7.978261436s" totalImagesPullingTime="369.241797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:52:14.976863931 +0000 UTC m=+2345.676627895" watchObservedRunningTime="2026-09-09 03:52:20.978261436 +0000 UTC m=+2351.678025399"
Sep 09 03:52:21.014524 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:21.014499    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"]
Sep 09 03:52:21.014747 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:21.014723    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7" podUID="ba48857f-4f0f-430e-b5ef-59e146ec4411" containerName="maas-api" containerID="cri-o://25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69" gracePeriod=30
Sep 09 03:52:22.264246 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.264225    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:52:22.371117 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.371040    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-maas-api-tls\") pod \"ba48857f-4f0f-430e-b5ef-59e146ec4411\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") "
Sep 09 03:52:22.371277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.371116    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs\") pod \"ba48857f-4f0f-430e-b5ef-59e146ec4411\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") "
Sep 09 03:52:22.371277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.371174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc62z\" (UniqueName: \"kubernetes.io/projected/ba48857f-4f0f-430e-b5ef-59e146ec4411-kube-api-access-dc62z\") pod \"ba48857f-4f0f-430e-b5ef-59e146ec4411\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") "
Sep 09 03:52:22.371277 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.371240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba48857f-4f0f-430e-b5ef-59e146ec4411-openshift-service-ca\") pod \"ba48857f-4f0f-430e-b5ef-59e146ec4411\" (UID: \"ba48857f-4f0f-430e-b5ef-59e146ec4411\") "
Sep 09 03:52:22.371612 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.371577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba48857f-4f0f-430e-b5ef-59e146ec4411-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba48857f-4f0f-430e-b5ef-59e146ec4411" (UID: "ba48857f-4f0f-430e-b5ef-59e146ec4411"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:52:22.373331 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.373303    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba48857f-4f0f-430e-b5ef-59e146ec4411" (UID: "ba48857f-4f0f-430e-b5ef-59e146ec4411"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:52:22.373435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.373364    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba48857f-4f0f-430e-b5ef-59e146ec4411-kube-api-access-dc62z" (OuterVolumeSpecName: "kube-api-access-dc62z") pod "ba48857f-4f0f-430e-b5ef-59e146ec4411" (UID: "ba48857f-4f0f-430e-b5ef-59e146ec4411"). InnerVolumeSpecName "kube-api-access-dc62z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:52:22.373435 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.373382    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ba48857f-4f0f-430e-b5ef-59e146ec4411" (UID: "ba48857f-4f0f-430e-b5ef-59e146ec4411"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:52:22.472128 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.472100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc62z\" (UniqueName: \"kubernetes.io/projected/ba48857f-4f0f-430e-b5ef-59e146ec4411-kube-api-access-dc62z\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:22.472128 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.472124    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba48857f-4f0f-430e-b5ef-59e146ec4411-openshift-service-ca\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:22.472278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.472133    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-maas-api-tls\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:22.472278 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.472142    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba48857f-4f0f-430e-b5ef-59e146ec4411-metrics-certs\") on node \"ip-10-0-142-178.ec2.internal\" DevicePath \"\""
Sep 09 03:52:22.996821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.996789    2580 generic.go:356] "Generic (PLEG): container finished" podID="ba48857f-4f0f-430e-b5ef-59e146ec4411" containerID="25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69" exitCode=0
Sep 09 03:52:22.997003 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.996828    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"
Sep 09 03:52:22.997003 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.996829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7" event={"ID":"ba48857f-4f0f-430e-b5ef-59e146ec4411","Type":"ContainerDied","Data":"25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69"}
Sep 09 03:52:22.997003 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.996971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7" event={"ID":"ba48857f-4f0f-430e-b5ef-59e146ec4411","Type":"ContainerDied","Data":"4976302cfea398c74a9c430b4ec745d26fd120eec5777f7617ef3c11995401d7"}
Sep 09 03:52:22.997188 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:22.997006    2580 scope.go:117] "RemoveContainer" containerID="25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69"
Sep 09 03:52:23.007142 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:23.007125    2580 scope.go:117] "RemoveContainer" containerID="25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69"
Sep 09 03:52:23.007406 ip-10-0-142-178 kubenswrapper[2580]: E0909 03:52:23.007389    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69\": container with ID starting with 25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69 not found: ID does not exist" containerID="25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69"
Sep 09 03:52:23.007448 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:23.007415    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69"} err="failed to get container status \"25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69\": rpc error: code = NotFound desc = could not find container \"25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69\": container with ID starting with 25d2bf606111e5aa270e0e869b6f5cbfa9ef732cf1bc5f24f93d8bef74556b69 not found: ID does not exist"
Sep 09 03:52:23.025904 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:23.025878    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"]
Sep 09 03:52:23.031821 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:23.031797    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d66fd5cd-9zdz7"]
Sep 09 03:52:23.789657 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:23.789629    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba48857f-4f0f-430e-b5ef-59e146ec4411" path="/var/lib/kubelet/pods/ba48857f-4f0f-430e-b5ef-59e146ec4411/volumes"
Sep 09 03:52:41.643774 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:41.643741    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vxd8h_de5c32ed-0c73-4bd5-8adc-5abfe87296f0/manager/0.log"
Sep 09 03:52:41.767266 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:41.767243    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nmsnb_1b7bd951-d54a-4aa3-afc4-92647ef56aca/manager/0.log"
Sep 09 03:52:42.060442 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:42.060419    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-d9dh9_6f5438cb-b06f-408f-8c7e-78269fdbafa6/server/0.log"
Sep 09 03:52:42.196332 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:42.196312    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rvt2k_a1176477-1719-48b3-b0b4-9e6a6020c315/manager/0.log"
Sep 09 03:52:42.310824 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:42.310776    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-nnbch_1deafbda-c739-4427-9487-66d90e13aecc/copy-manifests/0.log"
Sep 09 03:52:42.317987 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:42.317967    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-nnbch_1deafbda-c739-4427-9487-66d90e13aecc/manager/0.log"
Sep 09 03:52:42.936864 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:42.936829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6578d77458-ld4c8_9b2c2b99-cb6c-4513-ae7d-29e7df8df058/maas-api/0.log"
Sep 09 03:52:43.169785 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:43.169760    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815410-ldr56_60957e1c-8024-400f-a030-2fdda570b0fe/cleanup/0.log"
Sep 09 03:52:43.289040 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:43.289011    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815425-tttr7_427a0780-7a8e-4920-8714-f24301264c62/cleanup/0.log"
Sep 09 03:52:44.350351 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:44.350324    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-lppql_15e88706-5650-4175-aca6-21a55be6e0eb/manager/0.log"
Sep 09 03:52:44.459823 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:44.459794    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7clx8_dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2/manager/0.log"
Sep 09 03:52:44.695282 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:44.695222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-nrqvk_8a106ada-be9e-4e83-9cad-bd148837b33d/manager/0.log"
Sep 09 03:52:44.934258 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:44.934233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-bxfvm_5e75a48c-96cf-431e-a095-b0de7adc0039/manager/0.log"
Sep 09 03:52:45.287023 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:45.286994    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp_0493d4e1-8c33-4221-8152-a5d95f7eadc4/istio-proxy/0.log"
Sep 09 03:52:45.628967 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:45.628883    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f995d999-c62db_628ace00-06c2-4001-8132-018f8d6018a8/kube-auth-proxy/0.log"
Sep 09 03:52:45.749449 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:45.749405    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/1.log"
Sep 09 03:52:46.121156 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:46.121131    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fc79f467b-mq5wm_17db3411-56eb-4092-bd50-3ce1544e58e4/router/0.log"
Sep 09 03:52:46.472864 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:46.472834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bl446_af4807ff-d26b-4b20-b43e-48913075dd3b/main/0.log"
Sep 09 03:52:47.068393 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:47.068365    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-7lkbm_86233792-c185-4d21-bf09-6f660873f901/main/0.log"
Sep 09 03:52:47.197431 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:47.197403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8l4l4_55bff30a-8a10-4730-857b-851339e3266f/main/0.log"
Sep 09 03:52:56.561615 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:56.561584    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tx8l4_2d9cbca6-98ae-4c00-b9b6-c6368d560ff7/global-pull-secret-syncer/0.log"
Sep 09 03:52:56.762158 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:56.762135    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jwkkl_5bad8da5-211c-4a2a-adc6-130d5a3e7d9b/konnectivity-agent/0.log"
Sep 09 03:52:56.854651 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:52:56.854578    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-178.ec2.internal_8b9765c33d70e82b31ab7040648baa6e/haproxy/0.log"
Sep 09 03:53:01.205371 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:01.205339    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-lppql_15e88706-5650-4175-aca6-21a55be6e0eb/manager/0.log"
Sep 09 03:53:01.257443 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:01.257418    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7clx8_dfc7d2ce-165c-4f71-b5bf-3437dc6a6be2/manager/0.log"
Sep 09 03:53:01.387589 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:01.387560    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-nrqvk_8a106ada-be9e-4e83-9cad-bd148837b33d/manager/0.log"
Sep 09 03:53:01.498784 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:01.498759    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-bxfvm_5e75a48c-96cf-431e-a095-b0de7adc0039/manager/0.log"
Sep 09 03:53:03.557642 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:03.557611    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d774ddff9-6qgs6_0f6d3b4d-176d-4362-8b19-72568f95f938/metrics-server/0.log"
Sep 09 03:53:03.596722 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:03.596699    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-7kkqm_8dee1962-43ed-44c1-ac6c-7e5156757517/monitoring-plugin/0.log"
Sep 09 03:53:03.645550 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:03.645529    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-724z9_0dacdf99-bfd3-46cd-883a-0b2ee45499c8/node-exporter/0.log"
Sep 09 03:53:03.689021 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:03.688997    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-724z9_0dacdf99-bfd3-46cd-883a-0b2ee45499c8/kube-rbac-proxy/0.log"
Sep 09 03:53:03.718624 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:03.718603    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-724z9_0dacdf99-bfd3-46cd-883a-0b2ee45499c8/init-textfile/0.log"
Sep 09 03:53:04.778083 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.778049    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d4759d4b5-s9wkw_b2452083-20e5-4038-9739-b3c82ea97722/telemeter-client/0.log"
Sep 09 03:53:04.814898 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.814878    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d4759d4b5-s9wkw_b2452083-20e5-4038-9739-b3c82ea97722/reload/0.log"
Sep 09 03:53:04.842874 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.842853    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d4759d4b5-s9wkw_b2452083-20e5-4038-9739-b3c82ea97722/kube-rbac-proxy/0.log"
Sep 09 03:53:04.885982 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.885960    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-859fbb8db7-njr4l_059dee7b-d3d5-4309-afc3-678f05768f20/thanos-query/0.log"
Sep 09 03:53:04.921611 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.921589    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-859fbb8db7-njr4l_059dee7b-d3d5-4309-afc3-678f05768f20/kube-rbac-proxy-web/0.log"
Sep 09 03:53:04.957495 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.957474    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-859fbb8db7-njr4l_059dee7b-d3d5-4309-afc3-678f05768f20/kube-rbac-proxy/0.log"
Sep 09 03:53:04.989632 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:04.989610    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-859fbb8db7-njr4l_059dee7b-d3d5-4309-afc3-678f05768f20/prom-label-proxy/0.log"
Sep 09 03:53:05.023444 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:05.023425    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-859fbb8db7-njr4l_059dee7b-d3d5-4309-afc3-678f05768f20/kube-rbac-proxy-rules/0.log"
Sep 09 03:53:05.057214 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:05.057165    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-859fbb8db7-njr4l_059dee7b-d3d5-4309-afc3-678f05768f20/kube-rbac-proxy-metrics/0.log"
Sep 09 03:53:06.788735 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.788699    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"]
Sep 09 03:53:06.789260 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.789241    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba48857f-4f0f-430e-b5ef-59e146ec4411" containerName="maas-api"
Sep 09 03:53:06.789355 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.789263    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba48857f-4f0f-430e-b5ef-59e146ec4411" containerName="maas-api"
Sep 09 03:53:06.789447 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.789432    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba48857f-4f0f-430e-b5ef-59e146ec4411" containerName="maas-api"
Sep 09 03:53:06.792997 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.792975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:06.795516 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.795495    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qghgg\"/\"kube-root-ca.crt\""
Sep 09 03:53:06.796791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.796767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qghgg\"/\"default-dockercfg-cw55d\""
Sep 09 03:53:06.796791 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.796788    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qghgg\"/\"openshift-service-ca.crt\""
Sep 09 03:53:06.812869 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.812847    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"]
Sep 09 03:53:06.937795 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.937767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtf5w\" (UniqueName: \"kubernetes.io/projected/d2d4c03a-8dba-4481-8724-87b82edd0a20-kube-api-access-dtf5w\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:06.937918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.937805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-proc\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:06.937918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.937836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-lib-modules\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:06.937918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.937852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-podres\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:06.937918 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:06.937914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-sys\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038427 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-sys\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-sys\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtf5w\" (UniqueName: \"kubernetes.io/projected/d2d4c03a-8dba-4481-8724-87b82edd0a20-kube-api-access-dtf5w\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-proc\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038577 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-lib-modules\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-podres\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-proc\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-podres\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.038849 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.038742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2d4c03a-8dba-4481-8724-87b82edd0a20-lib-modules\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.047575 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.047557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtf5w\" (UniqueName: \"kubernetes.io/projected/d2d4c03a-8dba-4481-8724-87b82edd0a20-kube-api-access-dtf5w\") pod \"perf-node-gather-daemonset-k7vwc\" (UID: \"d2d4c03a-8dba-4481-8724-87b82edd0a20\") " pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.102809 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.102783    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:07.387965 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:07.387940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"]
Sep 09 03:53:07.390649 ip-10-0-142-178 kubenswrapper[2580]: W0909 03:53:07.390624    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd2d4c03a_8dba_4481_8724_87b82edd0a20.slice/crio-301d68d26823105e72fa4d097f6bb339f1ceb743089f751ba54c12f9a2cce01d WatchSource:0}: Error finding container 301d68d26823105e72fa4d097f6bb339f1ceb743089f751ba54c12f9a2cce01d: Status 404 returned error can't find the container with id 301d68d26823105e72fa4d097f6bb339f1ceb743089f751ba54c12f9a2cce01d
Sep 09 03:53:08.264743 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:08.264709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc" event={"ID":"d2d4c03a-8dba-4481-8724-87b82edd0a20","Type":"ContainerStarted","Data":"b6eafa9377b17ed62dc7f30e792b32b7c1bf0db3e51259e998a10d56b7b08426"}
Sep 09 03:53:08.265115 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:08.264764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc" event={"ID":"d2d4c03a-8dba-4481-8724-87b82edd0a20","Type":"ContainerStarted","Data":"301d68d26823105e72fa4d097f6bb339f1ceb743089f751ba54c12f9a2cce01d"}
Sep 09 03:53:08.265115 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:08.264788    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:08.291691 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:08.291628    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc" podStartSLOduration=2.291612554 podStartE2EDuration="2.291612554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:53:08.291298708 +0000 UTC m=+2398.991062670" watchObservedRunningTime="2026-09-09 03:53:08.291612554 +0000 UTC m=+2398.991376515"
Sep 09 03:53:09.528944 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:09.528915    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-52xfk_41de35a1-e1a6-41b7-8bfb-4a18889db59c/dns/0.log"
Sep 09 03:53:09.570889 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:09.570868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-52xfk_41de35a1-e1a6-41b7-8bfb-4a18889db59c/kube-rbac-proxy/0.log"
Sep 09 03:53:09.779060 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:09.778985    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdgpc_1ca41c40-7763-4d20-9e3a-6fd665bccb52/dns-node-resolver/0.log"
Sep 09 03:53:10.578550 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:10.578480    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zrhtb_99922913-bda7-4f39-9dbe-ee67f87431f9/node-ca/0.log"
Sep 09 03:53:11.661109 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:11.661080    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp_0493d4e1-8c33-4221-8152-a5d95f7eadc4/istio-proxy/0.log"
Sep 09 03:53:11.984036 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:11.984010    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f995d999-c62db_628ace00-06c2-4001-8132-018f8d6018a8/kube-auth-proxy/0.log"
Sep 09 03:53:12.029509 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:12.029479    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/0.log"
Sep 09 03:53:12.036528 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:12.036506    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/1.log"
Sep 09 03:53:12.188558 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:12.188535    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fc79f467b-mq5wm_17db3411-56eb-4092-bd50-3ce1544e58e4/router/0.log"
Sep 09 03:53:13.125429 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:13.125404    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gzpdr_92e422a5-0d5d-4642-86bf-e9f267ce6625/serve-healthcheck-canary/0.log"
Sep 09 03:53:13.826381 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:13.826354    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nm89b_11dde6d7-611d-4a45-99a7-c30ad55f00ca/kube-rbac-proxy/0.log"
Sep 09 03:53:13.856345 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:13.856317    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nm89b_11dde6d7-611d-4a45-99a7-c30ad55f00ca/exporter/0.log"
Sep 09 03:53:13.887878 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:13.887850    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nm89b_11dde6d7-611d-4a45-99a7-c30ad55f00ca/extractor/0.log"
Sep 09 03:53:14.280973 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:14.280952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qghgg/perf-node-gather-daemonset-k7vwc"
Sep 09 03:53:16.289842 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.289802    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vxd8h_de5c32ed-0c73-4bd5-8adc-5abfe87296f0/manager/0.log"
Sep 09 03:53:16.464732 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.464700    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nmsnb_1b7bd951-d54a-4aa3-afc4-92647ef56aca/manager/0.log"
Sep 09 03:53:16.558168 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.558072    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-d9dh9_6f5438cb-b06f-408f-8c7e-78269fdbafa6/server/0.log"
Sep 09 03:53:16.646169 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.646125    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rvt2k_a1176477-1719-48b3-b0b4-9e6a6020c315/manager/0.log"
Sep 09 03:53:16.677800 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.677781    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-nnbch_1deafbda-c739-4427-9487-66d90e13aecc/manager/0.log"
Sep 09 03:53:16.713762 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.713741    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff49c877-nnbch_1deafbda-c739-4427-9487-66d90e13aecc/copy-manifests/0.log"
Sep 09 03:53:16.996031 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:16.995999    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp_0493d4e1-8c33-4221-8152-a5d95f7eadc4/istio-proxy/0.log"
Sep 09 03:53:17.254046 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:17.254011    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f995d999-c62db_628ace00-06c2-4001-8132-018f8d6018a8/kube-auth-proxy/0.log"
Sep 09 03:53:17.292029 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:17.292004    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/0.log"
Sep 09 03:53:17.300518 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:17.300502    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/1.log"
Sep 09 03:53:17.377344 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:17.377311    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fc79f467b-mq5wm_17db3411-56eb-4092-bd50-3ce1544e58e4/router/0.log"
Sep 09 03:53:19.046251 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:19.046212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-fg8zp_0493d4e1-8c33-4221-8152-a5d95f7eadc4/istio-proxy/0.log"
Sep 09 03:53:19.323885 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:19.323803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f995d999-c62db_628ace00-06c2-4001-8132-018f8d6018a8/kube-auth-proxy/0.log"
Sep 09 03:53:19.364551 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:19.364522    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/0.log"
Sep 09 03:53:19.369431 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:19.369412    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cgtvx_d6dcaabd-8cb8-4537-861a-fc354c8f04e0/istio-proxy/1.log"
Sep 09 03:53:19.444750 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:19.444730    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fc79f467b-mq5wm_17db3411-56eb-4092-bd50-3ce1544e58e4/router/0.log"
Sep 09 03:53:23.739419 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:23.739378    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-nvjg5_30a07227-9188-4ada-8a0d-87ccbb6e728e/migrator/0.log"
Sep 09 03:53:23.768248 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:23.768224    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-nvjg5_30a07227-9188-4ada-8a0d-87ccbb6e728e/graceful-termination/0.log"
Sep 09 03:53:25.437133 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.437103    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/kube-multus-additional-cni-plugins/0.log"
Sep 09 03:53:25.464426 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.464387    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/egress-router-binary-copy/0.log"
Sep 09 03:53:25.492261 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.492239    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/cni-plugins/0.log"
Sep 09 03:53:25.517124 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.517102    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/bond-cni-plugin/0.log"
Sep 09 03:53:25.544204 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.544177    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/routeoverride-cni/0.log"
Sep 09 03:53:25.572959 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.572921    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/whereabouts-cni-bincopy/0.log"
Sep 09 03:53:25.600761 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.600735    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rmm89_96ed75e1-217e-49b7-939a-07720b22def2/whereabouts-cni/0.log"
Sep 09 03:53:25.884070 ip-10-0-142-178 kubenswrapper[2580]: I0909 03:53:25.884047    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lgf6v_db67cce9-62c6-45b2-b7af-24895bd4519b/kube-multus/0.log"