Jul 09 12:22:21.954539 ip-10-0-140-237 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 12:22:21.954553 ip-10-0-140-237 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 12:22:21.954563 ip-10-0-140-237 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 12:22:21.954884 ip-10-0-140-237 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 12:22:31.668351 ip-10-0-140-237 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 09 12:22:31.668367 ip-10-0-140-237 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6d73ad3db0c649f69c1907121881338f --
Jul 09 12:24:02.274988 ip-10-0-140-237 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.742035    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745920    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745932    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745936    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745939    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745942    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745947    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:24:02.777175 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745951    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745954    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745957    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745961    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745965    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745968    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745971    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745974    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745976    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745979    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745981    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745984    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745986    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745989    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745992    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745994    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.745997    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746000    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746002    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746005    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:24:03.030670 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746009    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:24:02.829113 ip-10-0-140-237 systemd[1]: Started Kubernetes Kubelet.
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746012    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746015    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746018    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746021    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746023    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746026    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746029    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746031    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746034    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746036    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746039    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746042    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746044    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746047    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746050    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746053    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746056    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746059    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746062    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:24:03.038509 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746064    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746067    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746070    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746074    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746077    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746080    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746084    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746087    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746089    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746092    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746094    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746097    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746099    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746102    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746105    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746108    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746110    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746113    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746117    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:24:03.039414 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746120    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746123    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746126    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746128    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746131    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746134    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746137    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746139    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746141    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746144    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746147    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746150    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746152    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746155    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746157    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746160    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746163    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746165    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746168    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746171    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746174    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:24:03.040078 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746177    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746180    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746183    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746185    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746187    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746190    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746193    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746196    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746198    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746201    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746203    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746206    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746208    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746961    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746968    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746972    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746975    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746978    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746981    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:24:03.040672 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746985    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746988    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746990    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746994    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746997    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.746999    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747003    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747005    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747008    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747011    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747013    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747017    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747020    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747022    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747025    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747028    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747032    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747036    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747040    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747043    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:24:03.041308 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747047    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747049    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747052    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747055    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747058    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747060    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747063    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747066    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747069    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747071    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747074    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747077    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747080    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747082    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747085    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747087    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747090    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747094    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747097    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747099    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:24:03.041936 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747102    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747104    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747107    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747110    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747113    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747116    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747118    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747121    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747124    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747126    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747129    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747133    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747137    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747139    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747145    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747147    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747150    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747153    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747156    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747158    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:24:03.042600 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747161    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747163    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747166    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747168    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747171    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747174    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747176    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747179    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747181    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747185    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747188    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747191    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747194    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747196    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747202    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747205    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747208    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747211    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747215    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:24:03.043355 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747218    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747220    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747223    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747225    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747228    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747230    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747233    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747237    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747239    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747242    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747245    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747247    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747249    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747252    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747628    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747643    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747646    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747649    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747653    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747656    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:24:03.044023 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747658    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747661    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747664    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747668    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747670    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747673    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747691    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747694    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747697    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747699    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747702    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747705    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747708    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747711    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747714    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747716    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747719    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747723    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747726    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747734    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:24:03.044733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747737    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747740    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747743    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747746    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747750    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747753    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747756    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747759    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747762    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747764    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747767    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747769    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747772    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747775    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747777    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747780    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747783    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747786    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747789    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:24:03.045434 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747792    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747794    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747799    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747801    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747804    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747806    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747809    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747812    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747814    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747817    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747819    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747822    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747827    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747829    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747832    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747835    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747837    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747840    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747842    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747845    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:24:03.046142 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747847    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747849    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747852    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747855    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747857    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747860    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747863    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747865    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747868    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747871    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747874    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747876    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747879    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747881    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747885    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747887    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747890    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747892    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747895    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747897    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:24:03.046938 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747900    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747902    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747905    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747907    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747911    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747914    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747916    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747919    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747921    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747924    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747926    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747929    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747931    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.747935    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748004    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748015    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748023    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748030    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748036    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748041    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748048    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 12:24:03.047931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748053    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748056    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748060    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748063    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748066    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748071    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748075    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748077    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748080    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748083    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748086    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748090    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748093    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748096    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748099    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748104    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748107    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748110    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748114    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748117    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748120    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748123    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748126    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748129    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 12:24:03.050994 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748132    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748136    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748139    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748142    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748145    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748148    2574 flags.go:64] FLAG: --enable-server="true"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748151    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748155    2574 flags.go:64] FLAG: --event-burst="100"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748158    2574 flags.go:64] FLAG: --event-qps="50"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748161    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748165    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748168    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748171    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748176    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748179    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748182    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748184    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748187    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748190    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748193    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748196    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748199    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748204    2574 flags.go:64] FLAG: --feature-gates=""
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748209    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748213    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 12:24:03.052658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748216    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748219    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748222    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748225    2574 flags.go:64] FLAG: --help="false"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748228    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748231    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748234    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748237    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748241    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748244    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748247    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748250    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748253    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748256    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748259    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748262    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748264    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748267    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748270    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748274    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748278    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748281    2574 flags.go:64] FLAG: --lock-file=""
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748284    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748287    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 12:24:03.053563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748290    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748295    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748298    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748301    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748304    2574 flags.go:64] FLAG: --logging-format="text"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748306    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748311    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748314    2574 flags.go:64] FLAG: --manifest-url=""
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748317    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748321    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748324    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748328    2574 flags.go:64] FLAG: --max-pods="110"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748331    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748334    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748337    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748340    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748343    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748345    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748348    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748355    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748358    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748361    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748364    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 09 12:24:03.055560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748366    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748372    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748375    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748378    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748381    2574 flags.go:64] FLAG: --port="10250"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748384    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748388    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-003dafcd134a4eaf6"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748392    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748395    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748397    2574 flags.go:64] FLAG: --register-node="true"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748400    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748403    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748406    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748409    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748412    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748420    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748423    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748426    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748429    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748432    2574 flags.go:64] FLAG: --runonce="false"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748435    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748438    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748441    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748444    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748446    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748449    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 12:24:03.057973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748453    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748456    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748458    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748461    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748464    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748467    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748469    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748472    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748475    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748481    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748483    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748486    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748491    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748494    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748496    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748499    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748502    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748505    2574 flags.go:64] FLAG: --v="2"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748508    2574 flags.go:64] FLAG: --version="false"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748512    2574 flags.go:64] FLAG: --vmodule=""
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748516    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.748520    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748639    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748642    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:24:03.064457 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748646    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748649    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748653    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748656    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748660    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748663    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748666    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748669    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748672    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748674    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748690    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748693    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748696    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748699    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748703    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748705    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748708    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748711    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748713    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:24:03.065303 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748716    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748720    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748723    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748726    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748729    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748731    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748734    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748737    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748739    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748742    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748747    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748750    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748752    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748755    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748757    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748760    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748763    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748765    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748768    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748771    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:24:03.283390 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748773    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748776    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748779    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748781    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748784    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748786    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748789    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748791    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748794    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748796    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748799    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748801    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748804    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748808    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748811    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748813    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748816    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748819    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748821    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748824    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:24:03.284398 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748826    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748829    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748833    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748835    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748838    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748840    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748843    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748852    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748857    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748860    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748863    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748865    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748869    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748873    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748876    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748879    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748882    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748885    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748888    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:24:03.285033 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748891    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748894    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748896    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748899    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748902    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748905    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748909    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748912    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748916    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748919    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748922    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748924    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748927    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748930    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748933    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748937    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748940    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748942    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:24:03.285571 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.748945    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.749878    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.761915    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.761934    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.761955    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.761964    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762027    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762033    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762036    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762040    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762043    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762046    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762048    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762051    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:24:03.286069 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762054    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762057    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762059    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762062    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762065    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762067    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762070    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762072    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762075    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762078    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762081    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762083    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762086    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762089    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762092    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762094    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762097    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762107    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762110    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762113    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:24:03.286499 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762116    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762118    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762121    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762124    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762127    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762129    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762132    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762135    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762137    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762140    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762143    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762146    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762149    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762152    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762154    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762157    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762159    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762164    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762168    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:24:03.287040 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762171    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762174    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762177    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762179    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762182    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762185    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762188    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762190    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762195    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762198    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762201    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762204    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762207    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762209    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762212    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762215    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762217    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762220    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762223    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762225    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:24:03.287566 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762228    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762230    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762237    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762240    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762243    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762245    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762249    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762254    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762257    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762259    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762262    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762265    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762268    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762271    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762276    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762279    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762281    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762284    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762287    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:24:03.288132 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762289    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762292    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762294    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762297    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762300    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762302    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762305    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762307    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762310    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762313    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762316    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762319    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762321    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.762327    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762464    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:24:03.288665 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762470    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762475    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762478    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762481    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762484    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762487    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762489    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762492    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762495    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762497    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762500    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762503    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762507    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762510    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762512    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762515    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762517    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762520    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762523    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762525    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:24:03.289138 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762528    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762531    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762533    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762536    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762539    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762541    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762544    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762546    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762549    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762552    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762555    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762557    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762560    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762564    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762566    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762569    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762571    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762574    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762577    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:24:03.289721 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762580    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762583    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762587    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762590    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762593    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762599    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762602    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762605    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762607    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762610    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762613    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762615    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762618    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762622    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762626    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762629    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762633    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762637    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762640    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:24:03.290249 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762644    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762647    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762651    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762655    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762658    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762662    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762666    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762671    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762674    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762693    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762697    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762701    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762705    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762710    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762713    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762717    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762721    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762724    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762729    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762734    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:24:03.290825 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762738    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762742    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762746    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762751    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762755    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762759    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762764    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762770    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762774    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762778    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762782    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762786    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762790    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762797    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762802    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762806    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762811    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762815    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762820    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:24:03.291338 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:02.762824    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.762830    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.762972    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.767300    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.768906    2574 server.go:1019] "Starting client certificate rotation"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.769011    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.769045    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.772537    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.777481    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.791518    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.794300    2574 log.go:25] "Validated CRI v1 image API"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.796090    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 12:24:03.291916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.798695    2574 fs.go:135] Filesystem UUIDs: map[6e05cbea-b71d-4f91-ad04-2bbe5b2ca19a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 87114c7e-b005-445b-8c0e-3347f3667462:/dev/nvme0n1p4]
Jul 09 12:24:03.292305 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.798718    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.803989    2574 manager.go:216] Machine: {Timestamp:2026-07-09 12:24:02.803283395 +0000 UTC m=+0.397907430 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099703 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23aeae898bc9b91020eaf0bc924ff3 SystemUUID:ec23aeae-898b-c9b9-1020-eaf0bc924ff3 BootID:6d73ad3d-b0c6-49f6-9c19-07121881338f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:8e:dc:a2:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:8e:dc:a2:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:3c:5b:c2:2b:5d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.804098    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.804187    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.806116    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.808101    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.808131    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-237.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 12:24:03.292408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.809674    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.809700    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.809708    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.810664    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.812518    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816628    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816758    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816787    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816797    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816809    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816818    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.816827    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.817235    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.817255    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.819903    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.822501    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.822528    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824134    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824149    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824158    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824163    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824172    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824178    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824188    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824193    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824200    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824206    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824214    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.824220    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.825038    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.825045    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.828139    2574 server.go:1268] "Started kubelet"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.828242    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.828315    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.828377    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.830247    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.830440    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.832322    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.832365    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 12:24:03.292663 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.832401    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.834949    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.836501    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6d36459d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.828109213 +0000 UTC m=+0.422733247,LastTimestamp:2026-07-09 12:24:02.828109213 +0000 UTC m=+0.422733247,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.842984    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843052    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.843070    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843742    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843755    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843901    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843919    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843906    2574 factory.go:55] Registering systemd factory
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.843949    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.844501    2574 factory.go:153] Registering CRI-O factory
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.844518    2574 factory.go:223] Registration of the crio container factory successfully
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.844579    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.844605    2574 factory.go:103] Registering Raw factory
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.844626    2574 manager.go:1196] Started watching for new ooms in manager
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.845151    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.845846    2574 manager.go:349] Starting recovery of all containers
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.846460    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.846461    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.857415    2574 manager.go:354] Recovery completed
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.862630    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.864426    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.864466    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.864477    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865045    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865057    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865071    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865072    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865741    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865772    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.865808    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.293759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.866151    2574 policy_none.go:49] "None policy: Start"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.866166    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.866178    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.867021    2574 policy_none.go:47] "Start"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.877955    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6f60c7e5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.864449509 +0000 UTC m=+0.459073543,LastTimestamp:2026-07-09 12:24:02.864449509 +0000 UTC m=+0.459073543,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.887145    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6f611c37  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.864471095 +0000 UTC m=+0.459095129,LastTimestamp:2026-07-09 12:24:02.864471095 +0000 UTC m=+0.459095129,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.900918    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6f614189  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.864480649 +0000 UTC m=+0.459104683,LastTimestamp:2026-07-09 12:24:02.864480649 +0000 UTC m=+0.459104683,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.907429    2574 manager.go:337] "Starting Device Plugin manager"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.907456    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.907465    2574 server.go:87] "Starting device plugin registration server"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.907735    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.907747    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.908020    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.908100    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.908109    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.909825    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 12:24:03.294904 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.909868    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.914149    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-237.ec2.internal.18c09e9b6f60c7e5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6f60c7e5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.864449509 +0000 UTC m=+0.459073543,LastTimestamp:2026-07-09 12:24:02.865755104 +0000 UTC m=+0.460379138,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.924647    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-237.ec2.internal.18c09e9b6f611c37\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6f611c37  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.864471095 +0000 UTC m=+0.459095129,LastTimestamp:2026-07-09 12:24:02.865779349 +0000 UTC m=+0.460403384,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.944128    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-237.ec2.internal.18c09e9b6f614189\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-237.ec2.internal.18c09e9b6f614189  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-237.ec2.internal,UID:ip-10-0-140-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-237.ec2.internal,},FirstTimestamp:2026-07-09 12:24:02.864480649 +0000 UTC m=+0.459104683,LastTimestamp:2026-07-09 12:24:02.865814663 +0000 UTC m=+0.460438697,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-237.ec2.internal,}"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.949048    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kxt8"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.961720    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kxt8"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.962883    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.964137    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.964180    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.964204    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:02.964241    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:02.975523    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.010786    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.011821    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.011856    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.011869    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.011893    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.295406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.026201    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.026221    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-237.ec2.internal\": node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.064908    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"]
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.065122    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.066126    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.066155    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.066166    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.067729    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.067846    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.067889    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.068460    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.068474    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.068484    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.068497    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.068498    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.068511    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.069623    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.069647    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.070283    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.070307    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.070322    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.088175    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.096416    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-237.ec2.internal\" not found" node="ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.100745    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-237.ec2.internal\" not found" node="ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.189235    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.245689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b1ad4ca39aed5d035e285e21716893f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal\" (UID: \"9b1ad4ca39aed5d035e285e21716893f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.494163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.245719    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b1ad4ca39aed5d035e285e21716893f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal\" (UID: \"9b1ad4ca39aed5d035e285e21716893f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.245737    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4828e88c979d58690bb239448cc9682f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-237.ec2.internal\" (UID: \"4828e88c979d58690bb239448cc9682f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.289874    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.346359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b1ad4ca39aed5d035e285e21716893f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal\" (UID: \"9b1ad4ca39aed5d035e285e21716893f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.346406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b1ad4ca39aed5d035e285e21716893f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal\" (UID: \"9b1ad4ca39aed5d035e285e21716893f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.346462    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b1ad4ca39aed5d035e285e21716893f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal\" (UID: \"9b1ad4ca39aed5d035e285e21716893f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.346464    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b1ad4ca39aed5d035e285e21716893f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal\" (UID: \"9b1ad4ca39aed5d035e285e21716893f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.346527    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4828e88c979d58690bb239448cc9682f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-237.ec2.internal\" (UID: \"4828e88c979d58690bb239448cc9682f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.346577    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4828e88c979d58690bb239448cc9682f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-237.ec2.internal\" (UID: \"4828e88c979d58690bb239448cc9682f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.390466    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.399618    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.404402    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:03.495557 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.491262    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.711957 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.591792    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.711957 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.649821    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:24:03.711957 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.692256    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.921368 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.767725    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 12:24:03.921368 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.767860    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 12:24:03.921368 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.767899    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 12:24:03.921368 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.793289    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:03.921368 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.820377    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:24:03.921368 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.893792    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.932335    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.933510    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.933553    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.933566    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.963630    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 12:19:03 +0000 UTC" deadline="2028-04-04 17:56:03.414946386 +0000 UTC"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:03.963661    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15245h31m59.451288608s"
Jul 09 12:24:04.076036 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:03.995078    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:04.333006 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.095911    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:04.333006 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:04.099788    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4828e88c979d58690bb239448cc9682f.slice/crio-2ace581fa62f6bcba6b42447fee07ac245625748a182f57043bde7d64903b568 WatchSource:0}: Error finding container 2ace581fa62f6bcba6b42447fee07ac245625748a182f57043bde7d64903b568: Status 404 returned error can't find the container with id 2ace581fa62f6bcba6b42447fee07ac245625748a182f57043bde7d64903b568
Jul 09 12:24:04.333006 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.106517    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:24:04.333006 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:04.114087    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1ad4ca39aed5d035e285e21716893f.slice/crio-99490926cd55837c03a90b345e26a3485717128b2d7c22409caa7cac5dd11866 WatchSource:0}: Error finding container 99490926cd55837c03a90b345e26a3485717128b2d7c22409caa7cac5dd11866: Status 404 returned error can't find the container with id 99490926cd55837c03a90b345e26a3485717128b2d7c22409caa7cac5dd11866
Jul 09 12:24:04.333006 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.198213    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:04.333006 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.298789    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-237.ec2.internal\" not found"
Jul 09 12:24:04.540429 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.380128    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:24:04.540429 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.444130    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:04.540429 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.459599    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 12:24:04.540429 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.460487    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal"
Jul 09 12:24:04.540429 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.469084    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.817476    2574 apiserver.go:52] "Watching apiserver"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.825513    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.825910    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-k2z55","openshift-ovn-kubernetes/ovnkube-node-cthvp","kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7k6lv","openshift-dns/node-resolver-5fnq2","openshift-multus/multus-additional-cni-plugins-6lqp8","openshift-multus/multus-x5mbg","openshift-network-operator/iptables-alerter-pm9mp","kube-system/global-pull-secret-syncer-vcrp2","kube-system/konnectivity-agent-w7vr7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2","openshift-image-registry/node-ca-xbjjh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal","openshift-multus/network-metrics-daemon-l72qj"]
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.828346    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.831444    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.837185    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.837193    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.837245    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.837653    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.837997    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.838335    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.839259    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.839274    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p5lks\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.839277    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.839295    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4m7vw\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.839377    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.840399    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.842690    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.843338    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fxwrv\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.843591    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.843935    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.844262    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.845132    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.846887    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f99st\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.846911    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.847027    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.861631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.847813    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.847905    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.848165    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.848770    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.849613    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h8zzf\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.851796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.854490    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.855807    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.856005    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mzc2g\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.857142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.858038    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.858403    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.858801    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.859280    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kt7dn\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.859528    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.859824    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860280    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.860345    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-systemd\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-os-release\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-conf-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860475    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-etc-kubernetes\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860502    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngqd8\" (UniqueName: \"kubernetes.io/projected/95d82425-10e6-41f6-b989-d73cbae5900f-kube-api-access-ngqd8\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860529    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860554    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-device-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-socket-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860676    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19366c30-84ae-4e90-a4bb-236550ef13c9-cni-binary-copy\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysctl-conf\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-run\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860774    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860755    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860853    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-var-lib-kubelet\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-cni-bin\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860935    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-system-cni-dir\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.865156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.860967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-registration-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861020    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-etc-selinux\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-cni-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-kubelet\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtc4w\" (UniqueName: \"kubernetes.io/projected/19366c30-84ae-4e90-a4bb-236550ef13c9-kube-api-access-xtc4w\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861193    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861218    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95d82425-10e6-41f6-b989-d73cbae5900f-etc-tuned\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95d82425-10e6-41f6-b989-d73cbae5900f-tmp\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-socket-dir-parent\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-cni-multus\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861336    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwb67\" (UniqueName: \"kubernetes.io/projected/ab0cfc02-fc44-4800-8624-37e26ac4f333-kube-api-access-fwb67\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861379    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f60c826-cf0e-4fe6-bb51-449024556554-dbus\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861415    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkp8x\" (UniqueName: \"kubernetes.io/projected/0c816e71-e2ed-4625-a51b-dc8c0142b081-kube-api-access-zkp8x\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861447    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cni-binary-copy\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.867390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861481    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60223d89-bcc2-4415-9dd6-c3bd15f1b115-hosts-file\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60223d89-bcc2-4415-9dd6-c3bd15f1b115-tmp-dir\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861556    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-multus-certs\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysconfig\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-lib-modules\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-kubernetes\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-netns\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-daemon-config\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysctl-d\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861804    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f60c826-cf0e-4fe6-bb51-449024556554-kubelet-config\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861861    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-cnibin\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.861961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-k8s-cni-cncf-io\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862075    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 12:24:04.868571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwk8z\" (UniqueName: \"kubernetes.io/projected/60223d89-bcc2-4415-9dd6-c3bd15f1b115-kube-api-access-zwk8z\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jchml\" (UniqueName: \"kubernetes.io/projected/5aa3ac72-32b5-4188-a6ca-72772cad5b90-kube-api-access-jchml\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-host\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-sys-fs\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-os-release\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-hostroot\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-modprobe-d\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862340    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-sys\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862319    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cnibin\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.862435    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-system-cni-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.864857    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.865115    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.865644    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dzhnr\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.865941    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.866129    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.866322    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.866445    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.866820    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-84kc2\""
Jul 09 12:24:04.870516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.869347    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.939825    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v7dg5"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.947040    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.950571    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v7dg5"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.962860    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-kubelet\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.962898    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-cni-bin\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.962925    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovnkube-config\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.962952    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovn-node-metrics-cert\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.962984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-os-release\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-conf-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-etc-kubernetes\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngqd8\" (UniqueName: \"kubernetes.io/projected/95d82425-10e6-41f6-b989-d73cbae5900f-kube-api-access-ngqd8\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963122    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963173    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-device-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.989291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-socket-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963253    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-slash\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-env-overrides\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963306    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4ndf\" (UniqueName: \"kubernetes.io/projected/a7f8befd-7746-416c-94a1-c78e5260a13a-kube-api-access-c4ndf\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19366c30-84ae-4e90-a4bb-236550ef13c9-cni-binary-copy\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysctl-conf\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-run\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-systemd\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-var-lib-kubelet\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963496    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-cni-bin\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963549    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-system-cni-dir\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963596    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-conf-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.992839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnscq\" (UniqueName: \"kubernetes.io/projected/6673b781-2ccd-4738-afe5-c1f1267ac18f-kube-api-access-fnscq\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-os-release\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963672    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-etc-kubernetes\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963718    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-system-cni-dir\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963748    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.963772    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-registration-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963827    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-run\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.963869    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:05.463836428 +0000 UTC m=+3.058460451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963880    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-cni-bin\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963892    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-etc-selinux\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-cni-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963929    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-var-lib-kubelet\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963944    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-socket-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.993728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-kubelet\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.963991    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-registration-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964024    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964087    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysctl-conf\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtc4w\" (UniqueName: \"kubernetes.io/projected/19366c30-84ae-4e90-a4bb-236550ef13c9-kube-api-access-xtc4w\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964119    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-etc-selinux\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964127    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-kubelet\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964160    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964206    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-device-dir\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95d82425-10e6-41f6-b989-d73cbae5900f-etc-tuned\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964250    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19366c30-84ae-4e90-a4bb-236550ef13c9-cni-binary-copy\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964262    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-cni-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.964274    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964290    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.996883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95d82425-10e6-41f6-b989-d73cbae5900f-tmp\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:04.964381    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:05.46436292 +0000 UTC m=+3.058986945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-socket-dir-parent\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964473    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-cni-multus\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwb67\" (UniqueName: \"kubernetes.io/projected/ab0cfc02-fc44-4800-8624-37e26ac4f333-kube-api-access-fwb67\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964544    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-socket-dir-parent\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964559    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-var-lib-cni-multus\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f60c826-cf0e-4fe6-bb51-449024556554-dbus\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964608    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964723    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkp8x\" (UniqueName: \"kubernetes.io/projected/0c816e71-e2ed-4625-a51b-dc8c0142b081-kube-api-access-zkp8x\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964757    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4f60c826-cf0e-4fe6-bb51-449024556554-dbus\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cni-binary-copy\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60223d89-bcc2-4415-9dd6-c3bd15f1b115-hosts-file\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.964958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60223d89-bcc2-4415-9dd6-c3bd15f1b115-tmp-dir\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:04.997820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60223d89-bcc2-4415-9dd6-c3bd15f1b115-hosts-file\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-multus-certs\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysconfig\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965211    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-lib-modules\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6673b781-2ccd-4738-afe5-c1f1267ac18f-host-slash\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-run-ovn-kubernetes\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965322    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cni-binary-copy\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965294    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60223d89-bcc2-4415-9dd6-c3bd15f1b115-tmp-dir\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965342    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysconfig\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965418    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965459    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-kubernetes\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/26cadada-4368-4c23-a200-d7f591eb9ce0-agent-certs\") pod \"konnectivity-agent-w7vr7\" (UID: \"26cadada-4368-4c23-a200-d7f591eb9ce0\") " pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965534    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7f8befd-7746-416c-94a1-c78e5260a13a-host\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.000043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965540    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-lib-modules\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965593    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-multus-certs\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-netns\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965600    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-kubernetes\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965657    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-netns\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-daemon-config\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysctl-d\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965807    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5aa3ac72-32b5-4188-a6ca-72772cad5b90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965876    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-sysctl-d\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f60c826-cf0e-4fe6-bb51-449024556554-kubelet-config\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965933    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-log-socket\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965969    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4f60c826-cf0e-4fe6-bb51-449024556554-kubelet-config\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.965981    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovnkube-script-lib\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:05.001192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966141    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19366c30-84ae-4e90-a4bb-236550ef13c9-multus-daemon-config\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7f8befd-7746-416c-94a1-c78e5260a13a-serviceca\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-cnibin\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966221    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-k8s-cni-cncf-io\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwk8z\" (UniqueName: \"kubernetes.io/projected/60223d89-bcc2-4415-9dd6-c3bd15f1b115-kube-api-access-zwk8z\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966261    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-cnibin\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jchml\" (UniqueName: \"kubernetes.io/projected/5aa3ac72-32b5-4188-a6ca-72772cad5b90-kube-api-access-jchml\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966302    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-ovn\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966308    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-host-run-k8s-cni-cncf-io\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966324    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-cni-netd\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966342    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-host\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-sys-fs\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966375    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-os-release\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6673b781-2ccd-4738-afe5-c1f1267ac18f-iptables-alerter-script\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.005143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-run-netns\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966407    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-host\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966457    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-os-release\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966480    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c816e71-e2ed-4625-a51b-dc8c0142b081-sys-fs\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-var-lib-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966516    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-node-log\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmwpw\" (UniqueName: \"kubernetes.io/projected/8dc34686-dcbc-4c12-b418-97c0f06e3af7-kube-api-access-tmwpw\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-hostroot\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-modprobe-d\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966613    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-sys\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966622    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-hostroot\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966635    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-systemd-units\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966663    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/26cadada-4368-4c23-a200-d7f591eb9ce0-konnectivity-ca\") pod \"konnectivity-agent-w7vr7\" (UID: \"26cadada-4368-4c23-a200-d7f591eb9ce0\") " pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966686    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cnibin\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966681    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-sys\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.006263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966701    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-modprobe-d\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-etc-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966740    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aa3ac72-32b5-4188-a6ca-72772cad5b90-cnibin\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-system-cni-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-systemd\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966838    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19366c30-84ae-4e90-a4bb-236550ef13c9-system-cni-dir\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.966862    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95d82425-10e6-41f6-b989-d73cbae5900f-etc-systemd\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.970645    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95d82425-10e6-41f6-b989-d73cbae5900f-tmp\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.974894    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95d82425-10e6-41f6-b989-d73cbae5900f-etc-tuned\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.977479    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal" event={"ID":"9b1ad4ca39aed5d035e285e21716893f","Type":"ContainerStarted","Data":"99490926cd55837c03a90b345e26a3485717128b2d7c22409caa7cac5dd11866"}
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.979393    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngqd8\" (UniqueName: \"kubernetes.io/projected/95d82425-10e6-41f6-b989-d73cbae5900f-kube-api-access-ngqd8\") pod \"tuned-7k6lv\" (UID: \"95d82425-10e6-41f6-b989-d73cbae5900f\") " pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.979903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal" event={"ID":"4828e88c979d58690bb239448cc9682f","Type":"ContainerStarted","Data":"2ace581fa62f6bcba6b42447fee07ac245625748a182f57043bde7d64903b568"}
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.984544    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwk8z\" (UniqueName: \"kubernetes.io/projected/60223d89-bcc2-4415-9dd6-c3bd15f1b115-kube-api-access-zwk8z\") pod \"node-resolver-5fnq2\" (UID: \"60223d89-bcc2-4415-9dd6-c3bd15f1b115\") " pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.986653    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtc4w\" (UniqueName: \"kubernetes.io/projected/19366c30-84ae-4e90-a4bb-236550ef13c9-kube-api-access-xtc4w\") pod \"multus-x5mbg\" (UID: \"19366c30-84ae-4e90-a4bb-236550ef13c9\") " pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.990146    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkp8x\" (UniqueName: \"kubernetes.io/projected/0c816e71-e2ed-4625-a51b-dc8c0142b081-kube-api-access-zkp8x\") pod \"aws-ebs-csi-driver-node-hjkr2\" (UID: \"0c816e71-e2ed-4625-a51b-dc8c0142b081\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:05.009009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.990390    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jchml\" (UniqueName: \"kubernetes.io/projected/5aa3ac72-32b5-4188-a6ca-72772cad5b90-kube-api-access-jchml\") pod \"multus-additional-cni-plugins-6lqp8\" (UID: \"5aa3ac72-32b5-4188-a6ca-72772cad5b90\") " pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.010140 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:04.991027    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwb67\" (UniqueName: \"kubernetes.io/projected/ab0cfc02-fc44-4800-8624-37e26ac4f333-kube-api-access-fwb67\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6673b781-2ccd-4738-afe5-c1f1267ac18f-host-slash\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-run-ovn-kubernetes\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067688    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/26cadada-4368-4c23-a200-d7f591eb9ce0-agent-certs\") pod \"konnectivity-agent-w7vr7\" (UID: \"26cadada-4368-4c23-a200-d7f591eb9ce0\") " pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067765    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6673b781-2ccd-4738-afe5-c1f1267ac18f-host-slash\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7f8befd-7746-416c-94a1-c78e5260a13a-host\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067817    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7f8befd-7746-416c-94a1-c78e5260a13a-host\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-log-socket\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067866    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-run-ovn-kubernetes\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovnkube-script-lib\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067908    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7f8befd-7746-416c-94a1-c78e5260a13a-serviceca\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.092361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067971    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-ovn\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.067996    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-cni-netd\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068024    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6673b781-2ccd-4738-afe5-c1f1267ac18f-iptables-alerter-script\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068048    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-run-netns\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-var-lib-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-node-log\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068110    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-log-socket\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068125    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmwpw\" (UniqueName: \"kubernetes.io/projected/8dc34686-dcbc-4c12-b418-97c0f06e3af7-kube-api-access-tmwpw\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068202    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-systemd-units\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/26cadada-4368-4c23-a200-d7f591eb9ce0-konnectivity-ca\") pod \"konnectivity-agent-w7vr7\" (UID: \"26cadada-4368-4c23-a200-d7f591eb9ce0\") " pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-etc-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-kubelet\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-cni-bin\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068352    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovnkube-config\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.093509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068368    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-ovn\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068377    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovn-node-metrics-cert\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068416    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-cni-netd\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-slash\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068458    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-env-overrides\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4ndf\" (UniqueName: \"kubernetes.io/projected/a7f8befd-7746-416c-94a1-c78e5260a13a-kube-api-access-c4ndf\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-systemd\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068529    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7f8befd-7746-416c-94a1-c78e5260a13a-serviceca\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068553    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-var-lib-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnscq\" (UniqueName: \"kubernetes.io/projected/6673b781-2ccd-4738-afe5-c1f1267ac18f-kube-api-access-fnscq\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068598    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-run-netns\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068601    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-node-log\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068660    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-systemd-units\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068794    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovnkube-script-lib\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.095158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068856    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-kubelet\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068909    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-cni-bin\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.068993    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6673b781-2ccd-4738-afe5-c1f1267ac18f-iptables-alerter-script\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069061    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069109    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-run-systemd\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069146    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-host-slash\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069194    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dc34686-dcbc-4c12-b418-97c0f06e3af7-etc-openvswitch\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069259    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovnkube-config\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069295    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/26cadada-4368-4c23-a200-d7f591eb9ce0-konnectivity-ca\") pod \"konnectivity-agent-w7vr7\" (UID: \"26cadada-4368-4c23-a200-d7f591eb9ce0\") " pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.069492    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dc34686-dcbc-4c12-b418-97c0f06e3af7-env-overrides\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.071830    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dc34686-dcbc-4c12-b418-97c0f06e3af7-ovn-node-metrics-cert\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.072154    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/26cadada-4368-4c23-a200-d7f591eb9ce0-agent-certs\") pod \"konnectivity-agent-w7vr7\" (UID: \"26cadada-4368-4c23-a200-d7f591eb9ce0\") " pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.074876    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.074903    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:05.096064 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.074922    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:05.096914 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.075030    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:24:05.574991887 +0000 UTC m=+3.169615912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:05.096914 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.077154    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4ndf\" (UniqueName: \"kubernetes.io/projected/a7f8befd-7746-416c-94a1-c78e5260a13a-kube-api-access-c4ndf\") pod \"node-ca-xbjjh\" (UID: \"a7f8befd-7746-416c-94a1-c78e5260a13a\") " pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.096914 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.079102    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmwpw\" (UniqueName: \"kubernetes.io/projected/8dc34686-dcbc-4c12-b418-97c0f06e3af7-kube-api-access-tmwpw\") pod \"ovnkube-node-cthvp\" (UID: \"8dc34686-dcbc-4c12-b418-97c0f06e3af7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.096914 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.079686    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnscq\" (UniqueName: \"kubernetes.io/projected/6673b781-2ccd-4738-afe5-c1f1267ac18f-kube-api-access-fnscq\") pod \"iptables-alerter-pm9mp\" (UID: \"6673b781-2ccd-4738-afe5-c1f1267ac18f\") " pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.146092    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x5mbg"
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.158701    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2"
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.175563    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7k6lv"
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.179399    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c816e71_e2ed_4625_a51b_dc8c0142b081.slice/crio-f6853069a79dba2eb3c5c574f7050cb9a4f79b7484a679f1e666b44aa35cd968 WatchSource:0}: Error finding container f6853069a79dba2eb3c5c574f7050cb9a4f79b7484a679f1e666b44aa35cd968: Status 404 returned error can't find the container with id f6853069a79dba2eb3c5c574f7050cb9a4f79b7484a679f1e666b44aa35cd968
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.179855    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5fnq2"
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.188564    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6lqp8"
Jul 09 12:24:05.205707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.199820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pm9mp"
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.209321    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d82425_10e6_41f6_b989_d73cbae5900f.slice/crio-b5cfd60c55859ea3c46cc0a14247811166bf3f96c1bea4675b389bdd3964d092 WatchSource:0}: Error finding container b5cfd60c55859ea3c46cc0a14247811166bf3f96c1bea4675b389bdd3964d092: Status 404 returned error can't find the container with id b5cfd60c55859ea3c46cc0a14247811166bf3f96c1bea4675b389bdd3964d092
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.210643    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60223d89_bcc2_4415_9dd6_c3bd15f1b115.slice/crio-5f9457220c1e34a580ac49ed667f929100c8fd7c31dfd12ec100ced930a1bc72 WatchSource:0}: Error finding container 5f9457220c1e34a580ac49ed667f929100c8fd7c31dfd12ec100ced930a1bc72: Status 404 returned error can't find the container with id 5f9457220c1e34a580ac49ed667f929100c8fd7c31dfd12ec100ced930a1bc72
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.216109    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.219174    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa3ac72_32b5_4188_a6ca_72772cad5b90.slice/crio-0c19470d713c9bf50eeb60a6573a2cd99bb68eba3635d925ed40ff7767108997 WatchSource:0}: Error finding container 0c19470d713c9bf50eeb60a6573a2cd99bb68eba3635d925ed40ff7767108997: Status 404 returned error can't find the container with id 0c19470d713c9bf50eeb60a6573a2cd99bb68eba3635d925ed40ff7767108997
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.222588    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.223245    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6673b781_2ccd_4738_afe5_c1f1267ac18f.slice/crio-ee6a1c61bfb89850cf0f2abbd0e4e57a466b84a388cfed771e983c678d6054e1 WatchSource:0}: Error finding container ee6a1c61bfb89850cf0f2abbd0e4e57a466b84a388cfed771e983c678d6054e1: Status 404 returned error can't find the container with id ee6a1c61bfb89850cf0f2abbd0e4e57a466b84a388cfed771e983c678d6054e1
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.228476    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xbjjh"
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.236038    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc34686_dcbc_4c12_b418_97c0f06e3af7.slice/crio-236e849d7a00c207ab436f6d2b8d0c6186e453401ad5ae90bec067c1324aa38e WatchSource:0}: Error finding container 236e849d7a00c207ab436f6d2b8d0c6186e453401ad5ae90bec067c1324aa38e: Status 404 returned error can't find the container with id 236e849d7a00c207ab436f6d2b8d0c6186e453401ad5ae90bec067c1324aa38e
Jul 09 12:24:05.267050 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:05.242014    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f8befd_7746_416c_94a1_c78e5260a13a.slice/crio-069fc5c37ab4a32b7dd6f8d439ea8f332319f223c71660b3de9d498f4728a298 WatchSource:0}: Error finding container 069fc5c37ab4a32b7dd6f8d439ea8f332319f223c71660b3de9d498f4728a298: Status 404 returned error can't find the container with id 069fc5c37ab4a32b7dd6f8d439ea8f332319f223c71660b3de9d498f4728a298
Jul 09 12:24:05.309880 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.309638    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:24:05.472286 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.472235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:05.472480 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.472308    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:05.472480 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.472404    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:05.472594 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.472480    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:05.472594 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.472493    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:06.472471662 +0000 UTC m=+4.067095690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:05.472594 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.472531    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:06.472516572 +0000 UTC m=+4.067140594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:05.674472 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.674382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:05.674620 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.674587    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:05.674620 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.674614    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:05.674687 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.674627    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:05.674745 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:05.674685    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:24:06.674669183 +0000 UTC m=+4.269293205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:05.955932 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.955883    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 12:19:05 +0000 UTC" deadline="2028-03-18 23:13:58.037220204 +0000 UTC"
Jul 09 12:24:05.955932 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.955930    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14842h49m52.081296001s"
Jul 09 12:24:05.983926 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.983815    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pm9mp" event={"ID":"6673b781-2ccd-4738-afe5-c1f1267ac18f","Type":"ContainerStarted","Data":"ee6a1c61bfb89850cf0f2abbd0e4e57a466b84a388cfed771e983c678d6054e1"}
Jul 09 12:24:05.985349 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.985305    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5fnq2" event={"ID":"60223d89-bcc2-4415-9dd6-c3bd15f1b115","Type":"ContainerStarted","Data":"5f9457220c1e34a580ac49ed667f929100c8fd7c31dfd12ec100ced930a1bc72"}
Jul 09 12:24:05.986894 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.986862    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2" event={"ID":"0c816e71-e2ed-4625-a51b-dc8c0142b081","Type":"ContainerStarted","Data":"f6853069a79dba2eb3c5c574f7050cb9a4f79b7484a679f1e666b44aa35cd968"}
Jul 09 12:24:05.988607 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.988553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x5mbg" event={"ID":"19366c30-84ae-4e90-a4bb-236550ef13c9","Type":"ContainerStarted","Data":"d3079a73f4f9f2d39db7ffe8f704e8d2be10e39ac77e31900978436d6bd773f6"}
Jul 09 12:24:05.989934 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.989903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"236e849d7a00c207ab436f6d2b8d0c6186e453401ad5ae90bec067c1324aa38e"}
Jul 09 12:24:05.991821 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.991782    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerStarted","Data":"0c19470d713c9bf50eeb60a6573a2cd99bb68eba3635d925ed40ff7767108997"}
Jul 09 12:24:05.993156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.993115    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7k6lv" event={"ID":"95d82425-10e6-41f6-b989-d73cbae5900f","Type":"ContainerStarted","Data":"b5cfd60c55859ea3c46cc0a14247811166bf3f96c1bea4675b389bdd3964d092"}
Jul 09 12:24:05.994554 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.994523    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xbjjh" event={"ID":"a7f8befd-7746-416c-94a1-c78e5260a13a","Type":"ContainerStarted","Data":"069fc5c37ab4a32b7dd6f8d439ea8f332319f223c71660b3de9d498f4728a298"}
Jul 09 12:24:05.995825 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:05.995797    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w7vr7" event={"ID":"26cadada-4368-4c23-a200-d7f591eb9ce0","Type":"ContainerStarted","Data":"c33f6dc42f7e151c35dda77838fcaf71b9edb270c1b3c6b1dec4872cd67b83eb"}
Jul 09 12:24:06.431409 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.431314    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:24:06.482472 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.482431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:06.482661 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.482495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:06.482661 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.482632    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:06.482790 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.482712    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:08.482690083 +0000 UTC m=+6.077314130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:06.482790 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.482633    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:06.482790 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.482783    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:08.482771021 +0000 UTC m=+6.077395045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:06.684837 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.684076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:06.684837 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.684269    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:06.684837 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.684295    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:06.684837 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.684309    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:06.684837 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.684373    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:24:08.684352281 +0000 UTC m=+6.278976318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:06.957022 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.956688    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 12:19:05 +0000 UTC" deadline="2028-01-05 04:01:27.500197591 +0000 UTC"
Jul 09 12:24:06.957022 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.956987    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13071h37m20.543217027s"
Jul 09 12:24:06.971644 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.970748    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:06.971644 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.970892    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:06.971644 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.971325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:06.971644 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.971438    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:06.971644 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:06.971504    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:06.971644 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:06.971594    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:07.023431 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:07.022570    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal" event={"ID":"4828e88c979d58690bb239448cc9682f","Type":"ContainerStarted","Data":"4fefc590e200eab76a0cc9af34a28ca3f9bc1b96bebc14a9c5c32c1d069ba8f2"}
Jul 09 12:24:08.040653 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.040598    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b1ad4ca39aed5d035e285e21716893f" containerID="e879bb3edb0ae7e24e53dde442687a9267fa4ab11483365e80c12d351b79c40d" exitCode=0
Jul 09 12:24:08.041610 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.041574    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal" event={"ID":"9b1ad4ca39aed5d035e285e21716893f","Type":"ContainerDied","Data":"e879bb3edb0ae7e24e53dde442687a9267fa4ab11483365e80c12d351b79c40d"}
Jul 09 12:24:08.061178 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.061120    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-237.ec2.internal" podStartSLOduration=4.061096567 podStartE2EDuration="4.061096567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:07.056627477 +0000 UTC m=+4.651251516" watchObservedRunningTime="2026-07-09 12:24:08.061096567 +0000 UTC m=+5.655720627"
Jul 09 12:24:08.501997 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.501957    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:08.502183 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.502026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:08.502248 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.502214    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:08.502321 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.502282    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:12.502262002 +0000 UTC m=+10.096886034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:08.502394 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.502371    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:08.502451 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.502419    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:12.502405783 +0000 UTC m=+10.097029819 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:08.706359 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.706317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:08.706599 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.706561    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:08.706599 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.706586    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:08.706599 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.706599    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:08.706846 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.706659    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:24:12.706640815 +0000 UTC m=+10.301264852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:08.965913 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.965102    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:08.965913 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.965243    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:08.965913 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.965634    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:08.965913 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.965735    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:08.965913 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:08.965799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:08.965913 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:08.965880    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:10.968525 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:10.968494    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:10.969021 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:10.968534    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:10.969021 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:10.968624    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:10.969021 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:10.968656    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:10.969021 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:10.968742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:10.969021 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:10.968865    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:12.543961 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:12.543316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:12.543961 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:12.543408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:12.543961 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.543527    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:12.543961 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.543595    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:20.543577803 +0000 UTC m=+18.138201824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:12.543961 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.543874    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:12.543961 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.543922    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:20.543906801 +0000 UTC m=+18.138530836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:12.744954 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:12.744450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:12.744954 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.744833    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:12.744954 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.744859    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:12.744954 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.744872    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:12.744954 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.744941    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:24:20.74492354 +0000 UTC m=+18.339547588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:12.966415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:12.966326    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:12.966582 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:12.966430    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:12.966582 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.966442    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:12.966582 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:12.966565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:12.966763 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.966584    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:12.966763 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:12.966664    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:14.965621 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:14.965586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:14.966090 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:14.965733    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:14.966153 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:14.966127    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:14.966269 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:14.966238    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:14.966342 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:14.966309    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:14.966408 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:14.966394    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:16.965921 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:16.965407    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:16.965921 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:16.965425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:16.965921 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:16.965425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:16.965921 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:16.965541    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:16.965921 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:16.965661    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:16.965921 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:16.965768    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:18.965134 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:18.965098    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:18.965575 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:18.965098    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:18.965575 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:18.965259    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:18.965575 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:18.965113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:18.965575 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:18.965323    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:18.965575 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:18.965422    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:20.602209 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:20.602168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:20.602728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:20.602225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:20.602728 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.602342    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:20.602728 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.602373    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:20.602728 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.602423    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:36.602401218 +0000 UTC m=+34.197025255 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:20.602728 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.602444    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:36.602433412 +0000 UTC m=+34.197057438 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:20.804189 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:20.804156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:20.804352 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.804290    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:20.804352 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.804306    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:20.804352 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.804318    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:20.804480 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.804379    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:24:36.804361357 +0000 UTC m=+34.398985379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:20.964811 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:20.964773    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:20.964993 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:20.964773    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:20.964993 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.964906    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:20.964993 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:20.964777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:20.965141 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.964985    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:20.965141 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:20.965085    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:22.967474 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:22.967442    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:22.967947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:22.967518    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:22.967947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:22.967649    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:22.967947 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:22.967666    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:22.967947 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:22.967783    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:22.967947 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:22.967863    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:24.965173 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:24.965007    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:24.965659 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:24.965249    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:24.965659 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:24.965310    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:24.965659 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:24.965374    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:24.965659 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:24.965431    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:24.965659 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:24.965512    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:25.081393 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.081359    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xbjjh" event={"ID":"a7f8befd-7746-416c-94a1-c78e5260a13a","Type":"ContainerStarted","Data":"00dc37b0ecb33f88965232c2c97bd9aa0cc3a3ef465e707904b55a38c942d6bd"}
Jul 09 12:24:25.082761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.082738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w7vr7" event={"ID":"26cadada-4368-4c23-a200-d7f591eb9ce0","Type":"ContainerStarted","Data":"d71cd3f82b5caf428a7e0f6c37d9fef1aa514fb7fff5644c65cac0df984bb0a7"}
Jul 09 12:24:25.084094 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.084075    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5fnq2" event={"ID":"60223d89-bcc2-4415-9dd6-c3bd15f1b115","Type":"ContainerStarted","Data":"5dd599e074bcd790158568c9f91285618b8d70c350e46b50356aaad1cb353f21"}
Jul 09 12:24:25.085405 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.085387    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2" event={"ID":"0c816e71-e2ed-4625-a51b-dc8c0142b081","Type":"ContainerStarted","Data":"08ac785ce38ce14765686518a36a4a90957df10fa8a345b4038f1ba109987bd5"}
Jul 09 12:24:25.086778 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.086758    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x5mbg" event={"ID":"19366c30-84ae-4e90-a4bb-236550ef13c9","Type":"ContainerStarted","Data":"b62a1abf4b42e74b7ee09779ae5c9c45a8a02c245215e9dd80682ac4dd687988"}
Jul 09 12:24:25.088532 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.088459    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal" event={"ID":"9b1ad4ca39aed5d035e285e21716893f","Type":"ContainerStarted","Data":"de0b6300bcde3d7163658a81e4071d828f457e3d3aae5f2d6e4f420d8db7cf56"}
Jul 09 12:24:25.090034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.090015    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"5fa5157a8d3f37d15377931a96f5e5a0474674069584d828ff5515e054a95a74"}
Jul 09 12:24:25.090113 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.090078    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"9913ccf965ce50f9cb6cb6656eabfea5fc42beca847c40996feeb3d75e736a23"}
Jul 09 12:24:25.093248 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.093226    2574 generic.go:356] "Generic (PLEG): container finished" podID="5aa3ac72-32b5-4188-a6ca-72772cad5b90" containerID="d9e9eb259eebf59e930fbeeaa1c90ddd68db8073ab4f62db255a298c8cc276fe" exitCode=0
Jul 09 12:24:25.093322 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.093285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerDied","Data":"d9e9eb259eebf59e930fbeeaa1c90ddd68db8073ab4f62db255a298c8cc276fe"}
Jul 09 12:24:25.094699 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.094664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7k6lv" event={"ID":"95d82425-10e6-41f6-b989-d73cbae5900f","Type":"ContainerStarted","Data":"c537bb8a53ca116328d2d082e8d108e04b8841188de54bc52299b24e0eae0555"}
Jul 09 12:24:25.096880 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.096839    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xbjjh" podStartSLOduration=2.757558571 podStartE2EDuration="22.096829105s" totalImagesPullingTime="19.339270534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:25.096741665 +0000 UTC m=+22.691365707" watchObservedRunningTime="2026-07-09 12:24:25.096829105 +0000 UTC m=+22.691453147"
Jul 09 12:24:25.114288 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.114247    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7k6lv" podStartSLOduration=2.713321338 podStartE2EDuration="22.1142343s" totalImagesPullingTime="19.400912962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:25.113803667 +0000 UTC m=+22.708427710" watchObservedRunningTime="2026-07-09 12:24:25.1142343 +0000 UTC m=+22.708858342"
Jul 09 12:24:25.168485 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.168445    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-237.ec2.internal" podStartSLOduration=21.168426898 podStartE2EDuration="21.168426898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:25.16829379 +0000 UTC m=+22.762917834" watchObservedRunningTime="2026-07-09 12:24:25.168426898 +0000 UTC m=+22.763050941"
Jul 09 12:24:25.185369 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.185330    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5fnq2" podStartSLOduration=3.16009203 podStartE2EDuration="22.185316233s" totalImagesPullingTime="19.025224203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:25.185134353 +0000 UTC m=+22.779758397" watchObservedRunningTime="2026-07-09 12:24:25.185316233 +0000 UTC m=+22.779940276"
Jul 09 12:24:25.220258 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.220218    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w7vr7" podStartSLOduration=2.875444496 podStartE2EDuration="22.220202619s" totalImagesPullingTime="19.344758123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:25.201929836 +0000 UTC m=+22.796553883" watchObservedRunningTime="2026-07-09 12:24:25.220202619 +0000 UTC m=+22.814826659"
Jul 09 12:24:25.220806 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:25.220778    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x5mbg" podStartSLOduration=2.770641081 podStartE2EDuration="22.220769968s" totalImagesPullingTime="19.450128887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:25.220331221 +0000 UTC m=+22.814955264" watchObservedRunningTime="2026-07-09 12:24:25.220769968 +0000 UTC m=+22.815394011"
Jul 09 12:24:26.100729 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.100662    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"becdb53aab38cb4a54c3e6a8f4ca3541066444602beb494c5345d59d030c5899"}
Jul 09 12:24:26.101260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.100823    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"99e29fe43d812c8dedd0fb5576cf4865495f9e23b7e514d856e9ebac84c8695a"}
Jul 09 12:24:26.101260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.100872    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"3f26650584dddeb024ef2a1ddb79f859cd0938b273e302713aa27c10268b3152"}
Jul 09 12:24:26.101260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.100922    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"0a6411a81d4ea537572a4e2f0fbcb73513ae769fe8d94ffc911a850194d96496"}
Jul 09 12:24:26.102924 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.102894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pm9mp" event={"ID":"6673b781-2ccd-4738-afe5-c1f1267ac18f","Type":"ContainerStarted","Data":"2a9243ea5eee3387fc6c9559efcd394ed603a38155e6cf50ca48a719a5c6d7c8"}
Jul 09 12:24:26.126606 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.126562    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pm9mp" podStartSLOduration=4.113375312 podStartE2EDuration="23.126546074s" totalImagesPullingTime="19.013170762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:26.126354629 +0000 UTC m=+23.720978670" watchObservedRunningTime="2026-07-09 12:24:26.126546074 +0000 UTC m=+23.721170116"
Jul 09 12:24:26.184162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.184134    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 12:24:26.921165 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.921065    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T12:24:26.18415326Z","UUID":"27c6726b-e914-4ad1-8ec2-8524036560b1","Handler":null,"Name":"","Endpoint":""}
Jul 09 12:24:26.922845 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.922818    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 12:24:26.922982 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.922881    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 12:24:26.958748 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.958716    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:26.959325 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.959305    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:26.964559 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.964540    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:26.964707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.964542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:26.964707 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:26.964658    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:26.964830 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:26.964797    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:26.964893 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:26.964864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:26.964977 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:26.964953    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:27.107699 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:27.107652    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2" event={"ID":"0c816e71-e2ed-4625-a51b-dc8c0142b081","Type":"ContainerStarted","Data":"19beb5e5941febb8d56d9791485c0e8bf487a9e2cd82528654ff674d9beea17c"}
Jul 09 12:24:27.108154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:27.107813    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:27.108527 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:27.108510    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w7vr7"
Jul 09 12:24:28.113204 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:28.113165    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"364cca6bd5d710f3c06823c12e48d7e0865c5fa8a68dde865839960f59ea46d8"}
Jul 09 12:24:28.115607 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:28.115575    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2" event={"ID":"0c816e71-e2ed-4625-a51b-dc8c0142b081","Type":"ContainerStarted","Data":"9b1bc40a07ddeaaa49396375a2b884c7b36ad0d5c1e12fe2480928b599cb9469"}
Jul 09 12:24:28.138136 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:28.138071    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hjkr2" podStartSLOduration=3.612298418 podStartE2EDuration="25.138054805s" totalImagesPullingTime="21.525756387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:28.137473219 +0000 UTC m=+25.732097290" watchObservedRunningTime="2026-07-09 12:24:28.138054805 +0000 UTC m=+25.732678896"
Jul 09 12:24:28.965495 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:28.965250    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:28.965700 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:28.965254    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:28.965700 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:28.965595    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:28.965700 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:28.965288    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:28.965700 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:28.965655    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:28.965860 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:28.965738    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:30.125433 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.125284    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" event={"ID":"8dc34686-dcbc-4c12-b418-97c0f06e3af7","Type":"ContainerStarted","Data":"96ec0b2026dd5e5003739ff7f39f53a3b4d8f69f52f78a7d58ef2f4f8aa6c859"}
Jul 09 12:24:30.126743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.126630    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:30.126743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.126730    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:30.126932 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.126780    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:30.143913 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.143886    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:30.144738 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.144718    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:24:30.171081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.171035    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp" podStartSLOduration=7.623697661 podStartE2EDuration="27.171020745s" totalImagesPullingTime="19.547323084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:30.170066351 +0000 UTC m=+27.764690418" watchObservedRunningTime="2026-07-09 12:24:30.171020745 +0000 UTC m=+27.765644836"
Jul 09 12:24:30.964978 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.964942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:30.965157 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.964942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:30.965157 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:30.965068    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:30.965157 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:30.965138    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:30.965157 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:30.964942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:30.965356 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:30.965211    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:32.129970 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.129889    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcrp2"]
Jul 09 12:24:32.130384 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.130023    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:32.130384 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:32.130105    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:32.131956 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.131934    2574 generic.go:356] "Generic (PLEG): container finished" podID="5aa3ac72-32b5-4188-a6ca-72772cad5b90" containerID="39cd3410a8da1ea925834bdedd8832df398acd873b124f07ccc693e0609be29d" exitCode=0
Jul 09 12:24:32.132059 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.132025    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerDied","Data":"39cd3410a8da1ea925834bdedd8832df398acd873b124f07ccc693e0609be29d"}
Jul 09 12:24:32.140724 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.140695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l72qj"]
Jul 09 12:24:32.140834 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.140821    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:32.140941 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:32.140923    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:32.143176 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.143151    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k2z55"]
Jul 09 12:24:32.143299 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:32.143285    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:32.143398 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:32.143368    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:33.136563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:33.136522    2574 generic.go:356] "Generic (PLEG): container finished" podID="5aa3ac72-32b5-4188-a6ca-72772cad5b90" containerID="6f3206cd70cc53914314077a3baa6d3828c962a5c6764bb01f2512584d53db0d" exitCode=0
Jul 09 12:24:33.136563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:33.136566    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerDied","Data":"6f3206cd70cc53914314077a3baa6d3828c962a5c6764bb01f2512584d53db0d"}
Jul 09 12:24:33.965301 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:33.965268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:33.965448 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:33.965286    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:33.965448 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:33.965424    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:33.965448 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:33.965419    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:33.965589 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:33.965497    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:33.965589 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:33.965560    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:34.141300 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:34.141207    2574 generic.go:356] "Generic (PLEG): container finished" podID="5aa3ac72-32b5-4188-a6ca-72772cad5b90" containerID="89d1feb0a31eb94035e66bd17c62ca4bbf752ea68f64bdc0f1bf93864469c4b8" exitCode=0
Jul 09 12:24:34.141300 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:34.141252    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerDied","Data":"89d1feb0a31eb94035e66bd17c62ca4bbf752ea68f64bdc0f1bf93864469c4b8"}
Jul 09 12:24:35.964772 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:35.964567    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:35.965243 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:35.964611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:35.965243 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:35.964867    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcrp2" podUID="4f60c826-cf0e-4fe6-bb51-449024556554"
Jul 09 12:24:35.965243 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:35.964618    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:35.965243 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:35.964926    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2z55" podUID="9c71e575-a551-497e-a4d3-1566de662efc"
Jul 09 12:24:35.965243 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:35.964996    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l72qj" podUID="ab0cfc02-fc44-4800-8624-37e26ac4f333"
Jul 09 12:24:36.621571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:36.621525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:36.621571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:36.621577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:36.621835 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.621709    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:36.621835 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.621729    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:36.621835 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.621762    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:08.621748399 +0000 UTC m=+66.216372420 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:24:36.621835 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.621794    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret podName:4f60c826-cf0e-4fe6-bb51-449024556554 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:08.621777205 +0000 UTC m=+66.216401240 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret") pod "global-pull-secret-syncer-vcrp2" (UID: "4f60c826-cf0e-4fe6-bb51-449024556554") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:24:36.822757 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:36.822716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:36.822922 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.822901    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:24:36.822979 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.822928    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:24:36.822979 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.822942    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rl588 for pod openshift-network-diagnostics/network-check-target-k2z55: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:36.823082 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:36.823009    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588 podName:9c71e575-a551-497e-a4d3-1566de662efc nodeName:}" failed. No retries permitted until 2026-07-09 12:25:08.822989983 +0000 UTC m=+66.417614024 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl588" (UniqueName: "kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588") pod "network-check-target-k2z55" (UID: "9c71e575-a551-497e-a4d3-1566de662efc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:24:37.753619 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.753590    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-237.ec2.internal" event="NodeReady"
Jul 09 12:24:37.754136 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.753758    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 12:24:37.793047 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.793019    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-644c647d7c-8c9gs"]
Jul 09 12:24:37.800903 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.800880    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.802610 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.802587    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7xf2k"]
Jul 09 12:24:37.803597 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.803575    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 12:24:37.803597 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.803589    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 12:24:37.803770 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.803596    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 12:24:37.803770 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.803596    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xhzlt\""
Jul 09 12:24:37.805011 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.804996    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:37.807451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.807430    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 12:24:37.807551 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.807486    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 12:24:37.808356 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.808298    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x78d8\""
Jul 09 12:24:37.809538 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.809518    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 12:24:37.809636 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.809607    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-scrxj"]
Jul 09 12:24:37.812443 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.812424    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644c647d7c-8c9gs"]
Jul 09 12:24:37.812544 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.812496    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:37.814627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.814608    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 12:24:37.814759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.814722    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 12:24:37.814825 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.814786    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qrtvq\""
Jul 09 12:24:37.814825 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.814821    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 12:24:37.817828 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.817810    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xf2k"]
Jul 09 12:24:37.821468 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.821450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-scrxj"]
Jul 09 12:24:37.931135 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931105    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnqcw\" (UniqueName: \"kubernetes.io/projected/53c9a2e7-219c-4889-8e51-e467dcef0381-kube-api-access-jnqcw\") pod \"ingress-canary-scrxj\" (UID: \"53c9a2e7-219c-4889-8e51-e467dcef0381\") " pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:37.931298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931141    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-trusted-ca\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931161    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-bound-sa-token\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53c9a2e7-219c-4889-8e51-e467dcef0381-cert\") pod \"ingress-canary-scrxj\" (UID: \"53c9a2e7-219c-4889-8e51-e467dcef0381\") " pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:37.931298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-image-registry-private-configuration\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931423 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-registry-certificates\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931423 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c060fbb9-db12-482e-8f9c-7fce74c25e48-tmp-dir\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:37.931423 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931423 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2cba3a95-1272-4252-ac50-51db912be7a9-ca-trust-extracted\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931423 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-installation-pull-secrets\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c060fbb9-db12-482e-8f9c-7fce74c25e48-config-volume\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:37.931580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931480    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmtlq\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-kube-api-access-gmtlq\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:37.931580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r7tq\" (UniqueName: \"kubernetes.io/projected/c060fbb9-db12-482e-8f9c-7fce74c25e48-kube-api-access-2r7tq\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:37.931580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.931529    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:37.965367 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.965334    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:24:37.965458 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.965435    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:24:37.965540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.965522    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:24:37.968276 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.968241    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 12:24:37.968442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.968304    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 12:24:37.968442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.968374    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 12:24:37.968442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.968312    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 12:24:37.968442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.968307    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x4nrr\""
Jul 09 12:24:37.968442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:37.968374    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9wl5j\""
Jul 09 12:24:38.032540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032453    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r7tq\" (UniqueName: \"kubernetes.io/projected/c060fbb9-db12-482e-8f9c-7fce74c25e48-kube-api-access-2r7tq\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.032540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032557    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnqcw\" (UniqueName: \"kubernetes.io/projected/53c9a2e7-219c-4889-8e51-e467dcef0381-kube-api-access-jnqcw\") pod \"ingress-canary-scrxj\" (UID: \"53c9a2e7-219c-4889-8e51-e467dcef0381\") " pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032587    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-trusted-ca\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-bound-sa-token\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53c9a2e7-219c-4889-8e51-e467dcef0381-cert\") pod \"ingress-canary-scrxj\" (UID: \"53c9a2e7-219c-4889-8e51-e467dcef0381\") " pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032719    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-image-registry-private-configuration\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-registry-certificates\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.032779 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c060fbb9-db12-482e-8f9c-7fce74c25e48-tmp-dir\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.032814    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2cba3a95-1272-4252-ac50-51db912be7a9-ca-trust-extracted\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-installation-pull-secrets\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c060fbb9-db12-482e-8f9c-7fce74c25e48-config-volume\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.032932    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls podName:c060fbb9-db12-482e-8f9c-7fce74c25e48 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:38.532912456 +0000 UTC m=+36.127536492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls") pod "dns-default-7xf2k" (UID: "c060fbb9-db12-482e-8f9c-7fce74c25e48") : secret "dns-default-metrics-tls" not found
Jul 09 12:24:38.033129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.032960    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmtlq\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-kube-api-access-gmtlq\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.033519 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.033494    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c060fbb9-db12-482e-8f9c-7fce74c25e48-tmp-dir\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.033664 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.033632    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-trusted-ca\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.033770 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.033724    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:24:38.033770 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.033745    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644c647d7c-8c9gs: secret "image-registry-tls" not found
Jul 09 12:24:38.033871 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.033832    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls podName:2cba3a95-1272-4252-ac50-51db912be7a9 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:38.533810258 +0000 UTC m=+36.128434293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls") pod "image-registry-644c647d7c-8c9gs" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9") : secret "image-registry-tls" not found
Jul 09 12:24:38.033871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.033844    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2cba3a95-1272-4252-ac50-51db912be7a9-ca-trust-extracted\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.034081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.034060    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-registry-certificates\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.034172 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.034152    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c060fbb9-db12-482e-8f9c-7fce74c25e48-config-volume\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.038040 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.038020    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-installation-pull-secrets\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.038040 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.038039    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53c9a2e7-219c-4889-8e51-e467dcef0381-cert\") pod \"ingress-canary-scrxj\" (UID: \"53c9a2e7-219c-4889-8e51-e467dcef0381\") " pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:38.038233 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.038019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-image-registry-private-configuration\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.042856 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.042834    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-bound-sa-token\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.043249 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.043222    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnqcw\" (UniqueName: \"kubernetes.io/projected/53c9a2e7-219c-4889-8e51-e467dcef0381-kube-api-access-jnqcw\") pod \"ingress-canary-scrxj\" (UID: \"53c9a2e7-219c-4889-8e51-e467dcef0381\") " pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:38.043740 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.043716    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r7tq\" (UniqueName: \"kubernetes.io/projected/c060fbb9-db12-482e-8f9c-7fce74c25e48-kube-api-access-2r7tq\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.044008 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.043988    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmtlq\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-kube-api-access-gmtlq\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.125964 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.125935    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-scrxj"
Jul 09 12:24:38.322747 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.322653    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-scrxj"]
Jul 09 12:24:38.539088 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.538873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:38.539255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:38.539132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:38.539255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.539035    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:24:38.539255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.539179    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644c647d7c-8c9gs: secret "image-registry-tls" not found
Jul 09 12:24:38.539255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.539248    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls podName:2cba3a95-1272-4252-ac50-51db912be7a9 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:39.539229692 +0000 UTC m=+37.133853713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls") pod "image-registry-644c647d7c-8c9gs" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9") : secret "image-registry-tls" not found
Jul 09 12:24:38.539465 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.539284    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:24:38.539465 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:38.539337    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls podName:c060fbb9-db12-482e-8f9c-7fce74c25e48 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:39.539323229 +0000 UTC m=+37.133947267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls") pod "dns-default-7xf2k" (UID: "c060fbb9-db12-482e-8f9c-7fce74c25e48") : secret "dns-default-metrics-tls" not found
Jul 09 12:24:39.547374 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:39.547330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:39.548210 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:39.547411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:39.548210 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:39.547522    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:24:39.548210 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:39.547545    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644c647d7c-8c9gs: secret "image-registry-tls" not found
Jul 09 12:24:39.548210 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:39.547576    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:24:39.548210 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:39.547608    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls podName:2cba3a95-1272-4252-ac50-51db912be7a9 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:41.547590378 +0000 UTC m=+39.142214401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls") pod "image-registry-644c647d7c-8c9gs" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9") : secret "image-registry-tls" not found
Jul 09 12:24:39.548210 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:39.547625    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls podName:c060fbb9-db12-482e-8f9c-7fce74c25e48 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:41.54761341 +0000 UTC m=+39.142237432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls") pod "dns-default-7xf2k" (UID: "c060fbb9-db12-482e-8f9c-7fce74c25e48") : secret "dns-default-metrics-tls" not found
Jul 09 12:24:39.874804 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:39.874718    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c9a2e7_219c_4889_8e51_e467dcef0381.slice/crio-0eb938fe36747e52e13e155e6a8fc33094062f0fdb8229c1760dcd027a335bd0 WatchSource:0}: Error finding container 0eb938fe36747e52e13e155e6a8fc33094062f0fdb8229c1760dcd027a335bd0: Status 404 returned error can't find the container with id 0eb938fe36747e52e13e155e6a8fc33094062f0fdb8229c1760dcd027a335bd0
Jul 09 12:24:40.159009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:40.158974    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-scrxj" event={"ID":"53c9a2e7-219c-4889-8e51-e467dcef0381","Type":"ContainerStarted","Data":"0eb938fe36747e52e13e155e6a8fc33094062f0fdb8229c1760dcd027a335bd0"}
Jul 09 12:24:41.163508 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:41.163469    2574 generic.go:356] "Generic (PLEG): container finished" podID="5aa3ac72-32b5-4188-a6ca-72772cad5b90" containerID="7e67deb7e2ecbe09476041ebb682a615955b5b16963a39f9876bcbdc33f4df77" exitCode=0
Jul 09 12:24:41.164020 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:41.163526    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerDied","Data":"7e67deb7e2ecbe09476041ebb682a615955b5b16963a39f9876bcbdc33f4df77"}
Jul 09 12:24:41.565146 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:41.565108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:41.565337 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:41.565210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:41.565337 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:41.565292    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:24:41.565445 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:41.565367    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls podName:c060fbb9-db12-482e-8f9c-7fce74c25e48 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:45.565344937 +0000 UTC m=+43.159968958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls") pod "dns-default-7xf2k" (UID: "c060fbb9-db12-482e-8f9c-7fce74c25e48") : secret "dns-default-metrics-tls" not found
Jul 09 12:24:41.565445 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:41.565369    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:24:41.565445 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:41.565386    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644c647d7c-8c9gs: secret "image-registry-tls" not found
Jul 09 12:24:41.565445 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:41.565442    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls podName:2cba3a95-1272-4252-ac50-51db912be7a9 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:45.565428747 +0000 UTC m=+43.160052770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls") pod "image-registry-644c647d7c-8c9gs" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9") : secret "image-registry-tls" not found
Jul 09 12:24:42.169216 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:42.169183    2574 generic.go:356] "Generic (PLEG): container finished" podID="5aa3ac72-32b5-4188-a6ca-72772cad5b90" containerID="9298ae8e2e2624352aa69a02e811578ce2130d6f0256c52fd571a5426270c976" exitCode=0
Jul 09 12:24:42.169624 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:42.169225    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerDied","Data":"9298ae8e2e2624352aa69a02e811578ce2130d6f0256c52fd571a5426270c976"}
Jul 09 12:24:43.176240 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:43.176202    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" event={"ID":"5aa3ac72-32b5-4188-a6ca-72772cad5b90","Type":"ContainerStarted","Data":"b1168885f476025cefe155251067bfc9b43e797c4ccfbdbacd128d63826f9c15"}
Jul 09 12:24:43.204864 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:43.204800    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6lqp8" podStartSLOduration=6.822443048 podStartE2EDuration="40.204780021s" totalImagesPullingTime="33.382336973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:43.204327691 +0000 UTC m=+40.798951735" watchObservedRunningTime="2026-07-09 12:24:43.204780021 +0000 UTC m=+40.799404065"
Jul 09 12:24:45.182396 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:45.182361    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-scrxj" event={"ID":"53c9a2e7-219c-4889-8e51-e467dcef0381","Type":"ContainerStarted","Data":"5caa3ebdd7f7f5a74a4e77d53c983ee0215a6ce8927fa35c8aad6aa24aec3f25"}
Jul 09 12:24:45.202584 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:45.202525    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-scrxj" podStartSLOduration=3.91913235 podStartE2EDuration="8.202505285s" totalImagesPullingTime="4.283372935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:24:45.202464478 +0000 UTC m=+42.797088533" watchObservedRunningTime="2026-07-09 12:24:45.202505285 +0000 UTC m=+42.797129323"
Jul 09 12:24:45.594029 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:45.593989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:45.594200 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:45.594143    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:24:45.594200 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:45.594159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:45.594274 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:45.594200    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls podName:c060fbb9-db12-482e-8f9c-7fce74c25e48 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:53.594184561 +0000 UTC m=+51.188808587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls") pod "dns-default-7xf2k" (UID: "c060fbb9-db12-482e-8f9c-7fce74c25e48") : secret "dns-default-metrics-tls" not found
Jul 09 12:24:45.594274 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:45.594250    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:24:45.594274 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:45.594262    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644c647d7c-8c9gs: secret "image-registry-tls" not found
Jul 09 12:24:45.594382 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:45.594299    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls podName:2cba3a95-1272-4252-ac50-51db912be7a9 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:53.594288253 +0000 UTC m=+51.188912274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls") pod "image-registry-644c647d7c-8c9gs" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9") : secret "image-registry-tls" not found
Jul 09 12:24:53.657273 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:53.657237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:24:53.657767 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:53.657296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:24:53.657767 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:53.657389    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:24:53.657767 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:53.657407    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644c647d7c-8c9gs: secret "image-registry-tls" not found
Jul 09 12:24:53.657767 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:53.657429    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:24:53.657767 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:53.657461    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls podName:2cba3a95-1272-4252-ac50-51db912be7a9 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:09.657446291 +0000 UTC m=+67.252070312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls") pod "image-registry-644c647d7c-8c9gs" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9") : secret "image-registry-tls" not found
Jul 09 12:24:53.657767 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:53.657474    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls podName:c060fbb9-db12-482e-8f9c-7fce74c25e48 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:09.65746828 +0000 UTC m=+67.252092300 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls") pod "dns-default-7xf2k" (UID: "c060fbb9-db12-482e-8f9c-7fce74c25e48") : secret "dns-default-metrics-tls" not found
Jul 09 12:24:56.338653 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.338611    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"]
Jul 09 12:24:56.349500 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.349468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.352394 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.352370    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:24:56.352394 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.352391    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 09 12:24:56.352600 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.352370    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 09 12:24:56.352600 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.352394    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6zmvr\""
Jul 09 12:24:56.353522 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.353499    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 09 12:24:56.354498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.354473    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"]
Jul 09 12:24:56.360119 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.360098    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"]
Jul 09 12:24:56.375275 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.375243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.375450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.375297    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-config\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.375450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.375428    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4fng\" (UniqueName: \"kubernetes.io/projected/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-kube-api-access-k4fng\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.381463 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.381442    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"]
Jul 09 12:24:56.381564 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.381539    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:56.384212 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.384190    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:24:56.384327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.384210    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 12:24:56.384327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.384216    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 12:24:56.384563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.384550    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pjf5q\""
Jul 09 12:24:56.438762 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.438726    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vvrdl"]
Jul 09 12:24:56.457573 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.457543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.459697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.459652    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vvrdl"]
Jul 09 12:24:56.460290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.460268    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:24:56.460413 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.460313    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 09 12:24:56.460482 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.460313    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7r66j\""
Jul 09 12:24:56.460692 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.460653    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 09 12:24:56.460832 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.460813    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 09 12:24:56.465815 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.465746    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 09 12:24:56.476061 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.476197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnp4v\" (UniqueName: \"kubernetes.io/projected/7041238b-5089-4f66-b150-6bf985194249-kube-api-access-nnp4v\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:56.476197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:56.476197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-trusted-ca\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.476197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-config\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.476197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476167    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-serving-cert\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.476197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gctvz\" (UniqueName: \"kubernetes.io/projected/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-kube-api-access-gctvz\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.476465 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476262    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4fng\" (UniqueName: \"kubernetes.io/projected/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-kube-api-access-k4fng\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.476465 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-config\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.476732 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.476714    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-config\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.479752 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.479735    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.491311 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.491289    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4fng\" (UniqueName: \"kubernetes.io/projected/d5395c69-4eb4-433d-a30b-387bcb6b2d8f-kube-api-access-k4fng\") pod \"service-ca-operator-5d46d9ffd4-jn6pb\" (UID: \"d5395c69-4eb4-433d-a30b-387bcb6b2d8f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.577529 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.577483    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gctvz\" (UniqueName: \"kubernetes.io/projected/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-kube-api-access-gctvz\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.577754 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.577573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-config\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.577754 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.577625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnp4v\" (UniqueName: \"kubernetes.io/projected/7041238b-5089-4f66-b150-6bf985194249-kube-api-access-nnp4v\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:56.577754 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.577729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:56.577924 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.577762    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-trusted-ca\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.577924 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.577801    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-serving-cert\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.577924 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:56.577869    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:24:56.578015 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:56.577950    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls podName:7041238b-5089-4f66-b150-6bf985194249 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:57.077930709 +0000 UTC m=+54.672554730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kzrp" (UID: "7041238b-5089-4f66-b150-6bf985194249") : secret "samples-operator-tls" not found
Jul 09 12:24:56.578354 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.578336    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-config\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.578555 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.578539    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-trusted-ca\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.580068 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.580053    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-serving-cert\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.586321 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.586296    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gctvz\" (UniqueName: \"kubernetes.io/projected/cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4-kube-api-access-gctvz\") pod \"console-operator-68b757fb67-vvrdl\" (UID: \"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4\") " pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.586433 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.586422    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnp4v\" (UniqueName: \"kubernetes.io/projected/7041238b-5089-4f66-b150-6bf985194249-kube-api-access-nnp4v\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:56.659452 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.659364    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"
Jul 09 12:24:56.769027 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.768998    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:24:56.782494 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.782460    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb"]
Jul 09 12:24:56.785974 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:56.785941    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5395c69_4eb4_433d_a30b_387bcb6b2d8f.slice/crio-94969d83e3ca07b2599cfda8e5401507c7dbbc7729688629271c2476ffce880d WatchSource:0}: Error finding container 94969d83e3ca07b2599cfda8e5401507c7dbbc7729688629271c2476ffce880d: Status 404 returned error can't find the container with id 94969d83e3ca07b2599cfda8e5401507c7dbbc7729688629271c2476ffce880d
Jul 09 12:24:56.894495 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:56.894464    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vvrdl"]
Jul 09 12:24:56.897630 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:24:56.897608    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc04df1_3e47_4c73_b3c8_54bd9aab3ab4.slice/crio-f27ed6eec3fa619e97172eb86de003bbebd11dd0f237c44708e1a620b930fef1 WatchSource:0}: Error finding container f27ed6eec3fa619e97172eb86de003bbebd11dd0f237c44708e1a620b930fef1: Status 404 returned error can't find the container with id f27ed6eec3fa619e97172eb86de003bbebd11dd0f237c44708e1a620b930fef1
Jul 09 12:24:57.082310 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:57.082281    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:57.082609 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:57.082451    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:24:57.082609 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:57.082538    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls podName:7041238b-5089-4f66-b150-6bf985194249 nodeName:}" failed. No retries permitted until 2026-07-09 12:24:58.082517888 +0000 UTC m=+55.677141920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kzrp" (UID: "7041238b-5089-4f66-b150-6bf985194249") : secret "samples-operator-tls" not found
Jul 09 12:24:57.210582 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:57.210544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" event={"ID":"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4","Type":"ContainerStarted","Data":"f27ed6eec3fa619e97172eb86de003bbebd11dd0f237c44708e1a620b930fef1"}
Jul 09 12:24:57.211586 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:57.211560    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb" event={"ID":"d5395c69-4eb4-433d-a30b-387bcb6b2d8f","Type":"ContainerStarted","Data":"94969d83e3ca07b2599cfda8e5401507c7dbbc7729688629271c2476ffce880d"}
Jul 09 12:24:58.090297 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:24:58.090257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:24:58.090784 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:58.090403    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:24:58.090784 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:24:58.090507    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls podName:7041238b-5089-4f66-b150-6bf985194249 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:00.090485056 +0000 UTC m=+57.685109082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kzrp" (UID: "7041238b-5089-4f66-b150-6bf985194249") : secret "samples-operator-tls" not found
Jul 09 12:25:00.105973 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.105936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:25:00.106359 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:00.106088    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:25:00.106359 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:00.106151    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls podName:7041238b-5089-4f66-b150-6bf985194249 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:04.106137209 +0000 UTC m=+61.700761230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kzrp" (UID: "7041238b-5089-4f66-b150-6bf985194249") : secret "samples-operator-tls" not found
Jul 09 12:25:00.221583 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.221553    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/0.log"
Jul 09 12:25:00.221785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.221597    2574 generic.go:356] "Generic (PLEG): container finished" podID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4" containerID="78d569dc0bfb6761dd45d0d8c024d4c23347c7006dd850d53f297c9932f4be13" exitCode=255
Jul 09 12:25:00.221785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.221663    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" event={"ID":"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4","Type":"ContainerDied","Data":"78d569dc0bfb6761dd45d0d8c024d4c23347c7006dd850d53f297c9932f4be13"}
Jul 09 12:25:00.221950 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.221934    2574 scope.go:117] "RemoveContainer" containerID="78d569dc0bfb6761dd45d0d8c024d4c23347c7006dd850d53f297c9932f4be13"
Jul 09 12:25:00.223256 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.223233    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb" event={"ID":"d5395c69-4eb4-433d-a30b-387bcb6b2d8f","Type":"ContainerStarted","Data":"e9259376c223625fe5aa1910c3927617c236fb8c358628154e080b21ffb2ba13"}
Jul 09 12:25:00.263884 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:00.263831    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb" podStartSLOduration=1.530825485 podStartE2EDuration="4.263813402s" totalImagesPullingTime="2.732987917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:00.262787832 +0000 UTC m=+57.857411903" watchObservedRunningTime="2026-07-09 12:25:00.263813402 +0000 UTC m=+57.858437444"
Jul 09 12:25:01.226733 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:01.226701    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/1.log"
Jul 09 12:25:01.227193 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:01.227178    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/0.log"
Jul 09 12:25:01.227249 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:01.227212    2574 generic.go:356] "Generic (PLEG): container finished" podID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4" containerID="bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4" exitCode=255
Jul 09 12:25:01.227323 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:01.227301    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" event={"ID":"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4","Type":"ContainerDied","Data":"bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4"}
Jul 09 12:25:01.227366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:01.227356    2574 scope.go:117] "RemoveContainer" containerID="78d569dc0bfb6761dd45d0d8c024d4c23347c7006dd850d53f297c9932f4be13"
Jul 09 12:25:01.227722 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:01.227669    2574 scope.go:117] "RemoveContainer" containerID="bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4"
Jul 09 12:25:01.227888 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:01.227869    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vvrdl_openshift-console-operator(cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4)\"" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4"
Jul 09 12:25:02.143290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:02.143257    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cthvp"
Jul 09 12:25:02.231454 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:02.231429    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/1.log"
Jul 09 12:25:02.231843 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:02.231823    2574 scope.go:117] "RemoveContainer" containerID="bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4"
Jul 09 12:25:02.232020 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:02.231998    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vvrdl_openshift-console-operator(cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4)\"" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4"
Jul 09 12:25:03.184012 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.183979    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-w5xn5"]
Jul 09 12:25:03.212634 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.212601    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-w5xn5"]
Jul 09 12:25:03.212801 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.212729    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.217530 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.217507    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 09 12:25:03.218442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.218411    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vzjrc\""
Jul 09 12:25:03.218602 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.218410    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 09 12:25:03.218602 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.218457    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 09 12:25:03.218602 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.218473    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 09 12:25:03.227669 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.227638    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9bb9\" (UniqueName: \"kubernetes.io/projected/49878e36-cf2d-4740-9604-ff2cf2a13c23-kube-api-access-d9bb9\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.227841 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.227698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/49878e36-cf2d-4740-9604-ff2cf2a13c23-signing-key\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.227841 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.227721    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/49878e36-cf2d-4740-9604-ff2cf2a13c23-signing-cabundle\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.328293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.328251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9bb9\" (UniqueName: \"kubernetes.io/projected/49878e36-cf2d-4740-9604-ff2cf2a13c23-kube-api-access-d9bb9\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.328717 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.328309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/49878e36-cf2d-4740-9604-ff2cf2a13c23-signing-key\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.328717 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.328328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/49878e36-cf2d-4740-9604-ff2cf2a13c23-signing-cabundle\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.329017 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.328998    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/49878e36-cf2d-4740-9604-ff2cf2a13c23-signing-cabundle\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.330575 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.330555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/49878e36-cf2d-4740-9604-ff2cf2a13c23-signing-key\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.340697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.340654    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9bb9\" (UniqueName: \"kubernetes.io/projected/49878e36-cf2d-4740-9604-ff2cf2a13c23-kube-api-access-d9bb9\") pod \"service-ca-758f599c5-w5xn5\" (UID: \"49878e36-cf2d-4740-9604-ff2cf2a13c23\") " pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.520912 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.520870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-w5xn5"
Jul 09 12:25:03.557319 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.557294    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6t9p2"]
Jul 09 12:25:03.575693 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.575644    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.576581 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.576522    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6t9p2"]
Jul 09 12:25:03.578315 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.578294    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6n9zx\""
Jul 09 12:25:03.578451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.578392    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 12:25:03.579182 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.579160    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 12:25:03.579292 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.579201    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 12:25:03.579292 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.579209    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 12:25:03.630240 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.630211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.630384 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.630244    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/16f7ba62-d0ff-4b6c-9281-b389135d478b-crio-socket\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.630498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.630471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkhlp\" (UniqueName: \"kubernetes.io/projected/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-api-access-wkhlp\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.630558 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.630544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/16f7ba62-d0ff-4b6c-9281-b389135d478b-data-volume\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.630624 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.630608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.683642 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.683613    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-w5xn5"]
Jul 09 12:25:03.686650 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:03.686620    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49878e36_cf2d_4740_9604_ff2cf2a13c23.slice/crio-57097dbc9c53a23fed1a13e76b05443b09b0b1f9d566b7952cace9fbe38cb5a0 WatchSource:0}: Error finding container 57097dbc9c53a23fed1a13e76b05443b09b0b1f9d566b7952cace9fbe38cb5a0: Status 404 returned error can't find the container with id 57097dbc9c53a23fed1a13e76b05443b09b0b1f9d566b7952cace9fbe38cb5a0
Jul 09 12:25:03.731924 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.731900    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkhlp\" (UniqueName: \"kubernetes.io/projected/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-api-access-wkhlp\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.732060 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.731952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/16f7ba62-d0ff-4b6c-9281-b389135d478b-data-volume\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.732060 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.731978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.732060 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.732012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.732060 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.732034    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/16f7ba62-d0ff-4b6c-9281-b389135d478b-crio-socket\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.732252 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:03.732130    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:25:03.732252 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.732143    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/16f7ba62-d0ff-4b6c-9281-b389135d478b-crio-socket\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.732252 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:03.732149    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:03.732252 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:03.732207    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:04.232191255 +0000 UTC m=+61.826815276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : configmap "kube-rbac-proxy" not found
Jul 09 12:25:03.732252 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:03.732226    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:04.232216712 +0000 UTC m=+61.826840734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:03.732468 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.732318    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/16f7ba62-d0ff-4b6c-9281-b389135d478b-data-volume\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:03.743327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:03.743301    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkhlp\" (UniqueName: \"kubernetes.io/projected/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-api-access-wkhlp\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:04.135641 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:04.135596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:25:04.135919 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:04.135757    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:25:04.135919 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:04.135815    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls podName:7041238b-5089-4f66-b150-6bf985194249 nodeName:}" failed. No retries permitted until 2026-07-09 12:25:12.135800638 +0000 UTC m=+69.730424660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9kzrp" (UID: "7041238b-5089-4f66-b150-6bf985194249") : secret "samples-operator-tls" not found
Jul 09 12:25:04.236039 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:04.236008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:04.236255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:04.236050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:04.236255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:04.236155    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:04.236255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:04.236175    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:25:04.236255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:04.236232    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:05.236217567 +0000 UTC m=+62.830841589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : configmap "kube-rbac-proxy" not found
Jul 09 12:25:04.236255 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:04.236248    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:05.236241574 +0000 UTC m=+62.830865595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:04.238731 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:04.238704    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-w5xn5" event={"ID":"49878e36-cf2d-4740-9604-ff2cf2a13c23","Type":"ContainerStarted","Data":"2785ddc3f455ab9e6639a9e7d5e12dde83d0b49692f717f7c67751ac4247e4af"}
Jul 09 12:25:04.238835 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:04.238749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-w5xn5" event={"ID":"49878e36-cf2d-4740-9604-ff2cf2a13c23","Type":"ContainerStarted","Data":"57097dbc9c53a23fed1a13e76b05443b09b0b1f9d566b7952cace9fbe38cb5a0"}
Jul 09 12:25:04.936911 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:04.936884    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5fnq2_60223d89-bcc2-4415-9dd6-c3bd15f1b115/dns-node-resolver/0.log"
Jul 09 12:25:05.245493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:05.245333    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:05.245493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:05.245390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:05.245755 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:05.245500    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:05.245755 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:05.245536    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:25:05.245755 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:05.245595    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:07.245557075 +0000 UTC m=+64.840181101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:05.245755 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:05.245622    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:07.245611663 +0000 UTC m=+64.840235689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : configmap "kube-rbac-proxy" not found
Jul 09 12:25:06.335295 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:06.335266    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xbjjh_a7f8befd-7746-416c-94a1-c78e5260a13a/node-ca/0.log"
Jul 09 12:25:06.732969 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:06.732932    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-scrxj_53c9a2e7-219c-4889-8e51-e467dcef0381/serve-healthcheck-canary/0.log"
Jul 09 12:25:06.769537 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:06.769493    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:25:06.769739 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:06.769551    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:25:06.769923 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:06.769910    2574 scope.go:117] "RemoveContainer" containerID="bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4"
Jul 09 12:25:06.770069 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:06.770051    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vvrdl_openshift-console-operator(cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4)\"" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4"
Jul 09 12:25:07.261546 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:07.261508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:07.261546 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:07.261554    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:07.261865 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:07.261655    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:25:07.261865 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:07.261676    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:07.261865 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:07.261737    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:11.261714417 +0000 UTC m=+68.856338443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : configmap "kube-rbac-proxy" not found
Jul 09 12:25:07.261865 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:07.261751    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:11.261744766 +0000 UTC m=+68.856368787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:08.672484 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.672450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:25:08.672908 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.672580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:25:08.675507 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.675488    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 12:25:08.675617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.675535    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 12:25:08.682911 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:08.682889    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 12:25:08.682974 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:08.682962    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs podName:ab0cfc02-fc44-4800-8624-37e26ac4f333 nodeName:}" failed. No retries permitted until 2026-07-09 12:26:12.682941306 +0000 UTC m=+130.277565344 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs") pod "network-metrics-daemon-l72qj" (UID: "ab0cfc02-fc44-4800-8624-37e26ac4f333") : secret "metrics-daemon-secret" not found
Jul 09 12:25:08.685042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.685016    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4f60c826-cf0e-4fe6-bb51-449024556554-original-pull-secret\") pod \"global-pull-secret-syncer-vcrp2\" (UID: \"4f60c826-cf0e-4fe6-bb51-449024556554\") " pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:25:08.874759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.874717    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:25:08.877711 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.877665    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 12:25:08.882721 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.882696    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcrp2"
Jul 09 12:25:08.888654 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.888631    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 12:25:08.898174 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:08.898144    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl588\" (UniqueName: \"kubernetes.io/projected/9c71e575-a551-497e-a4d3-1566de662efc-kube-api-access-rl588\") pod \"network-check-target-k2z55\" (UID: \"9c71e575-a551-497e-a4d3-1566de662efc\") " pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:25:09.022131 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.022071    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-w5xn5" podStartSLOduration=6.022049912 podStartE2EDuration="6.022049912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:04.260149246 +0000 UTC m=+61.854773290" watchObservedRunningTime="2026-07-09 12:25:09.022049912 +0000 UTC m=+66.616673955"
Jul 09 12:25:09.023183 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.023156    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcrp2"]
Jul 09 12:25:09.027016 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:09.026983    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f60c826_cf0e_4fe6_bb51_449024556554.slice/crio-895bfacea8b94b45fbdc64af3d6706d572593e5268b63b656965ca3c8fb8923e WatchSource:0}: Error finding container 895bfacea8b94b45fbdc64af3d6706d572593e5268b63b656965ca3c8fb8923e: Status 404 returned error can't find the container with id 895bfacea8b94b45fbdc64af3d6706d572593e5268b63b656965ca3c8fb8923e
Jul 09 12:25:09.177892 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.177855    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9wl5j\""
Jul 09 12:25:09.186099 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.186012    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:25:09.254189 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.254151    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcrp2" event={"ID":"4f60c826-cf0e-4fe6-bb51-449024556554","Type":"ContainerStarted","Data":"895bfacea8b94b45fbdc64af3d6706d572593e5268b63b656965ca3c8fb8923e"}
Jul 09 12:25:09.378839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.378807    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k2z55"]
Jul 09 12:25:09.382666 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:09.382623    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c71e575_a551_497e_a4d3_1566de662efc.slice/crio-39c3dbd9289150ebef5cc603a9d50a1a93ecfee23f4779b68fcc001ec0e91633 WatchSource:0}: Error finding container 39c3dbd9289150ebef5cc603a9d50a1a93ecfee23f4779b68fcc001ec0e91633: Status 404 returned error can't find the container with id 39c3dbd9289150ebef5cc603a9d50a1a93ecfee23f4779b68fcc001ec0e91633
Jul 09 12:25:09.681931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.681891    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:25:09.681931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.681942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:25:09.684340 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.684272    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c060fbb9-db12-482e-8f9c-7fce74c25e48-metrics-tls\") pod \"dns-default-7xf2k\" (UID: \"c060fbb9-db12-482e-8f9c-7fce74c25e48\") " pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:25:09.684340 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.684327    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"image-registry-644c647d7c-8c9gs\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") " pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:25:09.916866 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.916832    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xhzlt\""
Jul 09 12:25:09.922602 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.922574    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x78d8\""
Jul 09 12:25:09.924690 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.924654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:25:09.931079 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:09.931048    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:25:10.114624 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:10.114579    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644c647d7c-8c9gs"]
Jul 09 12:25:10.118879 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:10.118843    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cba3a95_1272_4252_ac50_51db912be7a9.slice/crio-04e4d049bc8bde99e70f5f0de8d6f3199d1dfb005d59dcb461f3b6d859ca1c8f WatchSource:0}: Error finding container 04e4d049bc8bde99e70f5f0de8d6f3199d1dfb005d59dcb461f3b6d859ca1c8f: Status 404 returned error can't find the container with id 04e4d049bc8bde99e70f5f0de8d6f3199d1dfb005d59dcb461f3b6d859ca1c8f
Jul 09 12:25:10.130544 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:10.130464    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xf2k"]
Jul 09 12:25:10.137143 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:10.135749    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc060fbb9_db12_482e_8f9c_7fce74c25e48.slice/crio-55e2014731d1615aac54edcc431b9839d0c4c3b80cfc83e45841d6b121467f4c WatchSource:0}: Error finding container 55e2014731d1615aac54edcc431b9839d0c4c3b80cfc83e45841d6b121467f4c: Status 404 returned error can't find the container with id 55e2014731d1615aac54edcc431b9839d0c4c3b80cfc83e45841d6b121467f4c
Jul 09 12:25:10.259047 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:10.259012    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xf2k" event={"ID":"c060fbb9-db12-482e-8f9c-7fce74c25e48","Type":"ContainerStarted","Data":"55e2014731d1615aac54edcc431b9839d0c4c3b80cfc83e45841d6b121467f4c"}
Jul 09 12:25:10.260568 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:10.260536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k2z55" event={"ID":"9c71e575-a551-497e-a4d3-1566de662efc","Type":"ContainerStarted","Data":"39c3dbd9289150ebef5cc603a9d50a1a93ecfee23f4779b68fcc001ec0e91633"}
Jul 09 12:25:10.263042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:10.262963    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" event={"ID":"2cba3a95-1272-4252-ac50-51db912be7a9","Type":"ContainerStarted","Data":"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"}
Jul 09 12:25:10.263042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:10.263013    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" event={"ID":"2cba3a95-1272-4252-ac50-51db912be7a9","Type":"ContainerStarted","Data":"04e4d049bc8bde99e70f5f0de8d6f3199d1dfb005d59dcb461f3b6d859ca1c8f"}
Jul 09 12:25:11.267577 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:11.267482    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:25:11.297713 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:11.296806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:11.297713 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:11.296872    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:11.297713 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:11.296958    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:25:11.297713 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:11.297014    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:19.296995646 +0000 UTC m=+76.891619668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : configmap "kube-rbac-proxy" not found
Jul 09 12:25:11.297713 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:11.297096    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:11.297713 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:11.297134    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls podName:16f7ba62-d0ff-4b6c-9281-b389135d478b nodeName:}" failed. No retries permitted until 2026-07-09 12:25:19.297122312 +0000 UTC m=+76.891746351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6t9p2" (UID: "16f7ba62-d0ff-4b6c-9281-b389135d478b") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:25:11.298521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:11.298460    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podStartSLOduration=67.298441128 podStartE2EDuration="1m7.298441128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:11.296869418 +0000 UTC m=+68.891493463" watchObservedRunningTime="2026-07-09 12:25:11.298441128 +0000 UTC m=+68.893065173"
Jul 09 12:25:12.208456 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:12.207790    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:25:12.217465 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:12.215819    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7041238b-5089-4f66-b150-6bf985194249-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9kzrp\" (UID: \"7041238b-5089-4f66-b150-6bf985194249\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:25:12.273787 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:12.273661    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xf2k" event={"ID":"c060fbb9-db12-482e-8f9c-7fce74c25e48","Type":"ContainerStarted","Data":"a933183a74d546b90ce043497283d09970c95c0ebc3d17f6ddb13a1b79de82c2"}
Jul 09 12:25:12.293478 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:12.293324    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pjf5q\""
Jul 09 12:25:12.300797 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:12.300735    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"
Jul 09 12:25:14.886460 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:14.886432    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp"]
Jul 09 12:25:15.285603 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.285561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcrp2" event={"ID":"4f60c826-cf0e-4fe6-bb51-449024556554","Type":"ContainerStarted","Data":"1331baf71a9a6533f1cfb65b4976712003411837b22555580e67b85306165eac"}
Jul 09 12:25:15.287371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.287341    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xf2k" event={"ID":"c060fbb9-db12-482e-8f9c-7fce74c25e48","Type":"ContainerStarted","Data":"363966be5e87942c4d09975d9f5ed057649d8947913f2605ee453e59403d0703"}
Jul 09 12:25:15.287522 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.287410    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:25:15.288911 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.288876    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k2z55" event={"ID":"9c71e575-a551-497e-a4d3-1566de662efc","Type":"ContainerStarted","Data":"1eb4386cc0a0254051b82aa2fc3f2fcc3ced1522a5c3b5ac86bb72aeb6eef897"}
Jul 09 12:25:15.289094 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.288921    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:25:15.289925 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.289903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp" event={"ID":"7041238b-5089-4f66-b150-6bf985194249","Type":"ContainerStarted","Data":"9c9acec6e1718b50ab15b2709c0358c014ac20a8ca727060b3c076d8fc074ed8"}
Jul 09 12:25:15.306850 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.306800    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcrp2" podStartSLOduration=66.594611534 podStartE2EDuration="1m12.306785925s" totalImagesPullingTime="5.712174391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:15.305810554 +0000 UTC m=+72.900434619" watchObservedRunningTime="2026-07-09 12:25:15.306785925 +0000 UTC m=+72.901409967"
Jul 09 12:25:15.323669 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.323619    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k2z55" podStartSLOduration=66.975615494 podStartE2EDuration="1m12.323604961s" totalImagesPullingTime="5.347989467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:15.323424154 +0000 UTC m=+72.918048216" watchObservedRunningTime="2026-07-09 12:25:15.323604961 +0000 UTC m=+72.918228983"
Jul 09 12:25:15.348795 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:15.348740    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7xf2k" podStartSLOduration=36.332357884 podStartE2EDuration="38.348721312s" totalImagesPullingTime="2.016363428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:15.347776945 +0000 UTC m=+72.942400990" watchObservedRunningTime="2026-07-09 12:25:15.348721312 +0000 UTC m=+72.943345356"
Jul 09 12:25:17.298604 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:17.298571    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp" event={"ID":"7041238b-5089-4f66-b150-6bf985194249","Type":"ContainerStarted","Data":"7cdf80251212f23cce9955d92e0dfb9fd0567d5f7dd1893fe0f26cf760e1f911"}
Jul 09 12:25:17.299027 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:17.298628    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp" event={"ID":"7041238b-5089-4f66-b150-6bf985194249","Type":"ContainerStarted","Data":"5d1e40b6ecc0b1847be9cbca2e63c9f01b63e7c93cbc5704f637f06dcf01c172"}
Jul 09 12:25:19.366004 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.365963    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:19.366004 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.366004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:19.366481 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.366464    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16f7ba62-d0ff-4b6c-9281-b389135d478b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:19.368262 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.368241    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16f7ba62-d0ff-4b6c-9281-b389135d478b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6t9p2\" (UID: \"16f7ba62-d0ff-4b6c-9281-b389135d478b\") " pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:19.486200 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.486147    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6t9p2"
Jul 09 12:25:19.616220 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.616099    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9kzrp" podStartSLOduration=21.687328327 podStartE2EDuration="23.616074225s" totalImagesPullingTime="1.928745898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:17.328890417 +0000 UTC m=+74.923514459" watchObservedRunningTime="2026-07-09 12:25:19.616074225 +0000 UTC m=+77.210698269"
Jul 09 12:25:19.616908 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:19.616884    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6t9p2"]
Jul 09 12:25:19.620309 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:19.620285    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f7ba62_d0ff_4b6c_9281_b389135d478b.slice/crio-4563306085d691bd02274e7c0791afe3b7bb4b51f4702aecf3035fa45ae462d2 WatchSource:0}: Error finding container 4563306085d691bd02274e7c0791afe3b7bb4b51f4702aecf3035fa45ae462d2: Status 404 returned error can't find the container with id 4563306085d691bd02274e7c0791afe3b7bb4b51f4702aecf3035fa45ae462d2
Jul 09 12:25:20.310558 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:20.310518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6t9p2" event={"ID":"16f7ba62-d0ff-4b6c-9281-b389135d478b","Type":"ContainerStarted","Data":"4563306085d691bd02274e7c0791afe3b7bb4b51f4702aecf3035fa45ae462d2"}
Jul 09 12:25:20.965164 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:20.965134    2574 scope.go:117] "RemoveContainer" containerID="bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4"
Jul 09 12:25:21.315603 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:21.315526    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:25:21.316052 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:21.316036    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/1.log"
Jul 09 12:25:21.316154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:21.316071    2574 generic.go:356] "Generic (PLEG): container finished" podID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4" containerID="3431b43a2c1cee411c49691a4c29f323be99b0df4e9bb32e73c45725563ce5d5" exitCode=255
Jul 09 12:25:21.316154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:21.316116    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" event={"ID":"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4","Type":"ContainerDied","Data":"3431b43a2c1cee411c49691a4c29f323be99b0df4e9bb32e73c45725563ce5d5"}
Jul 09 12:25:21.316259 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:21.316164    2574 scope.go:117] "RemoveContainer" containerID="bc4ddd8f464cafabe6e6ccfba01c3db80f1964e0f14f94ccc1fb5c9ad9e202d4"
Jul 09 12:25:21.316491 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:21.316472    2574 scope.go:117] "RemoveContainer" containerID="3431b43a2c1cee411c49691a4c29f323be99b0df4e9bb32e73c45725563ce5d5"
Jul 09 12:25:21.316741 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:21.316716    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-vvrdl_openshift-console-operator(cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4)\"" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4"
Jul 09 12:25:22.296549 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:22.296516    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7xf2k"
Jul 09 12:25:24.333136 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:24.333087    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:25:24.334775 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:24.334743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6t9p2" event={"ID":"16f7ba62-d0ff-4b6c-9281-b389135d478b","Type":"ContainerStarted","Data":"0d421d89b2f2b52f562fe1dba8ef120f3f02d9512107454e9dbc79fa1bdc2b56"}
Jul 09 12:25:24.334902 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:24.334821    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6t9p2" event={"ID":"16f7ba62-d0ff-4b6c-9281-b389135d478b","Type":"ContainerStarted","Data":"ce11fe0b514cda219034f7aedb1eb1a3711223b1d0df26e8d84acba3fd121cc4"}
Jul 09 12:25:26.769610 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:26.769574    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:25:26.770044 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:26.769632    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:25:26.770044 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:26.769945    2574 scope.go:117] "RemoveContainer" containerID="3431b43a2c1cee411c49691a4c29f323be99b0df4e9bb32e73c45725563ce5d5"
Jul 09 12:25:26.770122 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:26.770081    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-vvrdl_openshift-console-operator(cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4)\"" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4"
Jul 09 12:25:27.347032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:27.346993    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6t9p2" event={"ID":"16f7ba62-d0ff-4b6c-9281-b389135d478b","Type":"ContainerStarted","Data":"9292d9ad640190936e4d4d91ad466dc2b339ae7d67f3b8363b56961d6e4a1eea"}
Jul 09 12:25:27.370521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:27.370470    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6t9p2" podStartSLOduration=17.457953754 podStartE2EDuration="24.370452501s" totalImagesPullingTime="6.912498747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:25:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:27.369536561 +0000 UTC m=+84.964160595" watchObservedRunningTime="2026-07-09 12:25:27.370452501 +0000 UTC m=+84.965076544"
Jul 09 12:25:29.929654 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:29.929618    2574 patch_prober.go:28] interesting pod/image-registry-644c647d7c-8c9gs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:25:29.930040 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:29.929667    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:25:32.279661 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:32.279623    2574 patch_prober.go:28] interesting pod/image-registry-644c647d7c-8c9gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:25:32.280120 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:32.279713    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:25:37.964886 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:37.964855    2574 scope.go:117] "RemoveContainer" containerID="3431b43a2c1cee411c49691a4c29f323be99b0df4e9bb32e73c45725563ce5d5"
Jul 09 12:25:37.965327 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:25:37.965015    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-vvrdl_openshift-console-operator(cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4)\"" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4"
Jul 09 12:25:39.929114 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:39.929080    2574 patch_prober.go:28] interesting pod/image-registry-644c647d7c-8c9gs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:25:39.929490 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:39.929156    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:25:42.278688 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:42.278634    2574 patch_prober.go:28] interesting pod/image-registry-644c647d7c-8c9gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:25:42.279044 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:42.278704    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:25:46.295904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:46.295875    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k2z55"
Jul 09 12:25:49.327964 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.327935    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"]
Jul 09 12:25:49.331772 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.331749    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.334290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.334257    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 09 12:25:49.335128 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.335110    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 09 12:25:49.335249 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.335164    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sm44v\""
Jul 09 12:25:49.354948 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.354913    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"]
Jul 09 12:25:49.436661 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.436626    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"]
Jul 09 12:25:49.439655 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.439630    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"]
Jul 09 12:25:49.439803 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.439769    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:49.441391 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.441372    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.448876 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.448842    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 09 12:25:49.448876 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.448861    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 09 12:25:49.449067 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.448976    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 12:25:49.449067 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.448851    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gzvmp\""
Jul 09 12:25:49.449219 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.449174    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 09 12:25:49.449361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.449341    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 12:25:49.449439 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.449420    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 12:25:49.449482 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.449457    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 09 12:25:49.449482 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.449459    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 12:25:49.461513 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.461486    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"]
Jul 09 12:25:49.485699 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.485648    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"]
Jul 09 12:25:49.504760 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.504729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e7f1626-e51b-4256-b4d9-86c4097246b6-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-v57sx\" (UID: \"0e7f1626-e51b-4256-b4d9-86c4097246b6\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.504944 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.504838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e7f1626-e51b-4256-b4d9-86c4097246b6-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-v57sx\" (UID: \"0e7f1626-e51b-4256-b4d9-86c4097246b6\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.605755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxd9m\" (UniqueName: \"kubernetes.io/projected/f11aed02-e524-4252-a389-b61a4313ea55-kube-api-access-mxd9m\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.605755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.605755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.606034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605763    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e7f1626-e51b-4256-b4d9-86c4097246b6-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-v57sx\" (UID: \"0e7f1626-e51b-4256-b4d9-86c4097246b6\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.606034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-ca\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.606034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3729668-4a62-4863-9f85-fe9ce337689b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h4zzl\" (UID: \"f3729668-4a62-4863-9f85-fe9ce337689b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:49.606034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f11aed02-e524-4252-a389-b61a4313ea55-additional-service-ca\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.606034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605855    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e7f1626-e51b-4256-b4d9-86c4097246b6-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-v57sx\" (UID: \"0e7f1626-e51b-4256-b4d9-86c4097246b6\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.606034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.605871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-hub\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.606403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.606383    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e7f1626-e51b-4256-b4d9-86c4097246b6-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-v57sx\" (UID: \"0e7f1626-e51b-4256-b4d9-86c4097246b6\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.608363 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.608340    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0e7f1626-e51b-4256-b4d9-86c4097246b6-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-v57sx\" (UID: \"0e7f1626-e51b-4256-b4d9-86c4097246b6\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.641327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.641285    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"
Jul 09 12:25:49.707166 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxd9m\" (UniqueName: \"kubernetes.io/projected/f11aed02-e524-4252-a389-b61a4313ea55-kube-api-access-mxd9m\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.707300 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.707300 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.707562 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-ca\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.707604 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3729668-4a62-4863-9f85-fe9ce337689b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h4zzl\" (UID: \"f3729668-4a62-4863-9f85-fe9ce337689b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:49.707665 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f11aed02-e524-4252-a389-b61a4313ea55-additional-service-ca\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.707743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.707725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-hub\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.708497 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.708443    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f11aed02-e524-4252-a389-b61a4313ea55-additional-service-ca\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.709950 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.709920    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3729668-4a62-4863-9f85-fe9ce337689b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h4zzl\" (UID: \"f3729668-4a62-4863-9f85-fe9ce337689b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:49.710413 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.710378    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-ca\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.710413 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.710395    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.710574 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.710528    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.711052 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.711035    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f11aed02-e524-4252-a389-b61a4313ea55-hub\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.724040 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.724014    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxd9m\" (UniqueName: \"kubernetes.io/projected/f11aed02-e524-4252-a389-b61a4313ea55-kube-api-access-mxd9m\") pod \"cluster-proxy-proxy-agent-689b866fcb-grg88\" (UID: \"f11aed02-e524-4252-a389-b61a4313ea55\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.751827 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.751793    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:49.770361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.770335    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"
Jul 09 12:25:49.803305 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.803273    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-v57sx"]
Jul 09 12:25:49.807562 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:49.807534    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7f1626_e51b_4256_b4d9_86c4097246b6.slice/crio-7df9167eb27146b45f1e57b2b8a9159625f3b3ce240ed2baed7a0c0ff6478ecb WatchSource:0}: Error finding container 7df9167eb27146b45f1e57b2b8a9159625f3b3ce240ed2baed7a0c0ff6478ecb: Status 404 returned error can't find the container with id 7df9167eb27146b45f1e57b2b8a9159625f3b3ce240ed2baed7a0c0ff6478ecb
Jul 09 12:25:49.909107 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.909075    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"]
Jul 09 12:25:49.911747 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:49.911724    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3729668_4a62_4863_9f85_fe9ce337689b.slice/crio-bf6b07de0cd1cd0bbdfcd4e830511659bdc702d211a269ab3311e4158828dd43 WatchSource:0}: Error finding container bf6b07de0cd1cd0bbdfcd4e830511659bdc702d211a269ab3311e4158828dd43: Status 404 returned error can't find the container with id bf6b07de0cd1cd0bbdfcd4e830511659bdc702d211a269ab3311e4158828dd43
Jul 09 12:25:49.926462 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.926430    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88"]
Jul 09 12:25:49.929242 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.929217    2574 patch_prober.go:28] interesting pod/image-registry-644c647d7c-8c9gs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:25:49.929342 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.929276    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:25:49.929342 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.929327    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:25:49.929733 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:49.929704    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf11aed02_e524_4252_a389_b61a4313ea55.slice/crio-11ca9df5d9d29db6756fb81ac19e40e87ae8fcbc9dec03af754970a50a51be28 WatchSource:0}: Error finding container 11ca9df5d9d29db6756fb81ac19e40e87ae8fcbc9dec03af754970a50a51be28: Status 404 returned error can't find the container with id 11ca9df5d9d29db6756fb81ac19e40e87ae8fcbc9dec03af754970a50a51be28
Jul 09 12:25:49.929858 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.929816    2574 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"} pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:25:49.933418 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.933396    2574 patch_prober.go:28] interesting pod/image-registry-644c647d7c-8c9gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:25:49.933506 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:49.933434    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:25:50.423129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:50.423090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl" event={"ID":"f3729668-4a62-4863-9f85-fe9ce337689b","Type":"ContainerStarted","Data":"bf6b07de0cd1cd0bbdfcd4e830511659bdc702d211a269ab3311e4158828dd43"}
Jul 09 12:25:50.424134 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:50.424114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88" event={"ID":"f11aed02-e524-4252-a389-b61a4313ea55","Type":"ContainerStarted","Data":"11ca9df5d9d29db6756fb81ac19e40e87ae8fcbc9dec03af754970a50a51be28"}
Jul 09 12:25:50.425091 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:50.425071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx" event={"ID":"0e7f1626-e51b-4256-b4d9-86c4097246b6","Type":"ContainerStarted","Data":"7df9167eb27146b45f1e57b2b8a9159625f3b3ce240ed2baed7a0c0ff6478ecb"}
Jul 09 12:25:51.965258 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:51.965225    2574 scope.go:117] "RemoveContainer" containerID="3431b43a2c1cee411c49691a4c29f323be99b0df4e9bb32e73c45725563ce5d5"
Jul 09 12:25:52.436493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.436453    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx" event={"ID":"0e7f1626-e51b-4256-b4d9-86c4097246b6","Type":"ContainerStarted","Data":"4ecd0456e4136d339928788006d2440707e72fbbdcfee6a1df1c1988fe3d1044"}
Jul 09 12:25:52.438722 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.438670    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:25:52.438842 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.438816    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" event={"ID":"cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4","Type":"ContainerStarted","Data":"2f06c0376153782618c3af31d4d560ba84438394de388ec9dd833baaa13fef7d"}
Jul 09 12:25:52.439199 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.439103    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:25:52.440596 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.440525    2574 patch_prober.go:28] interesting pod/console-operator-68b757fb67-vvrdl container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.11:8443/readyz\": dial tcp 10.132.0.11:8443: connect: connection refused" start-of-body=
Jul 09 12:25:52.440596 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.440569    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl" event={"ID":"f3729668-4a62-4863-9f85-fe9ce337689b","Type":"ContainerStarted","Data":"911c22e54aac87a8e8c91f944693cc808e8bee96944a485bc2f5b5a82a360489"}
Jul 09 12:25:52.440596 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.440575    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podUID="cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.11:8443/readyz\": dial tcp 10.132.0.11:8443: connect: connection refused"
Jul 09 12:25:52.441029 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.441005    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:52.446474 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.446446    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl"
Jul 09 12:25:52.477935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.477870    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-v57sx" podStartSLOduration=0.955432767 podStartE2EDuration="3.477849903s" totalImagesPullingTime="2.522417136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:52.477520622 +0000 UTC m=+110.072144685" watchObservedRunningTime="2026-07-09 12:25:52.477849903 +0000 UTC m=+110.072473948"
Jul 09 12:25:52.505390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.505332    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl" podStartSLOduration=53.886450565 podStartE2EDuration="56.50531118s" totalImagesPullingTime="2.618860615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:52.50413777 +0000 UTC m=+110.098761877" watchObservedRunningTime="2026-07-09 12:25:52.50531118 +0000 UTC m=+110.099935227"
Jul 09 12:25:52.528772 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:52.528657    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h4zzl" podStartSLOduration=1.11178098 podStartE2EDuration="3.528636412s" totalImagesPullingTime="2.416855432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:52.525274624 +0000 UTC m=+110.119898668" watchObservedRunningTime="2026-07-09 12:25:52.528636412 +0000 UTC m=+110.123260455"
Jul 09 12:25:53.715862 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.715804    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-vvrdl"
Jul 09 12:25:53.955585 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.955550    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-fgmgf"]
Jul 09 12:25:53.960497 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.960474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:25:53.963182 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.963154    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 12:25:53.963488 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.963465    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 12:25:53.963673 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.963657    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ccplh\""
Jul 09 12:25:53.976208 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:53.976176    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-fgmgf"]
Jul 09 12:25:54.046054 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.046019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f9xh\" (UniqueName: \"kubernetes.io/projected/92c091f5-c366-46fe-9012-9a05a80a2898-kube-api-access-4f9xh\") pod \"downloads-685bfb5f64-fgmgf\" (UID: \"92c091f5-c366-46fe-9012-9a05a80a2898\") " pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:25:54.147350 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.147306    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f9xh\" (UniqueName: \"kubernetes.io/projected/92c091f5-c366-46fe-9012-9a05a80a2898-kube-api-access-4f9xh\") pod \"downloads-685bfb5f64-fgmgf\" (UID: \"92c091f5-c366-46fe-9012-9a05a80a2898\") " pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:25:54.156194 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.156164    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f9xh\" (UniqueName: \"kubernetes.io/projected/92c091f5-c366-46fe-9012-9a05a80a2898-kube-api-access-4f9xh\") pod \"downloads-685bfb5f64-fgmgf\" (UID: \"92c091f5-c366-46fe-9012-9a05a80a2898\") " pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:25:54.270261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.270173    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:25:54.451980 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.451939    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88" event={"ID":"f11aed02-e524-4252-a389-b61a4313ea55","Type":"ContainerStarted","Data":"dec11a8d65519f828a1b47168797cda5749beb55ed9ff41428ba0131fdd15125"}
Jul 09 12:25:54.452164 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.451997    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88" event={"ID":"f11aed02-e524-4252-a389-b61a4313ea55","Type":"ContainerStarted","Data":"ca8503d76d614a36e81becee7e6c345b0e0b3ad03bb9e31bff7f33420051badb"}
Jul 09 12:25:54.452164 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.452017    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88" event={"ID":"f11aed02-e524-4252-a389-b61a4313ea55","Type":"ContainerStarted","Data":"ee1aa13bb0d907898a9ef69bfdd9d98cd53b10bd3b990370cd8706fb45d76b16"}
Jul 09 12:25:54.489894 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.489825    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-689b866fcb-grg88" podStartSLOduration=1.85141352 podStartE2EDuration="5.489802618s" totalImagesPullingTime="3.638389098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:25:54.476935343 +0000 UTC m=+112.071559387" watchObservedRunningTime="2026-07-09 12:25:54.489802618 +0000 UTC m=+112.084426659"
Jul 09 12:25:54.490585 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:54.490563    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-fgmgf"]
Jul 09 12:25:54.494158 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:25:54.494137    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c091f5_c366_46fe_9012_9a05a80a2898.slice/crio-aacbaae046aaba98a16f5f4a81c479781139104c391abe16672bd0787ef2de1d WatchSource:0}: Error finding container aacbaae046aaba98a16f5f4a81c479781139104c391abe16672bd0787ef2de1d: Status 404 returned error can't find the container with id aacbaae046aaba98a16f5f4a81c479781139104c391abe16672bd0787ef2de1d
Jul 09 12:25:55.457503 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:55.457455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-fgmgf" event={"ID":"92c091f5-c366-46fe-9012-9a05a80a2898","Type":"ContainerStarted","Data":"aacbaae046aaba98a16f5f4a81c479781139104c391abe16672bd0787ef2de1d"}
Jul 09 12:25:59.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.925330    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rkxnv"]
Jul 09 12:25:59.933810 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.933781    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.936573 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.936547    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:25:59.937053 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937031    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 12:25:59.937177 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937059    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 12:25:59.937382 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937237    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 12:25:59.937992 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937960    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 12:25:59.937992 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937984    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 12:25:59.938199 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937960    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h44bc\""
Jul 09 12:25:59.938199 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.937972    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 12:25:59.997990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.997748    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.997990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.997813    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-textfile\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.997990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.997838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.997990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.997874    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-tls\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.997990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.997905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsbr5\" (UniqueName: \"kubernetes.io/projected/9e97cb6e-3ac8-4e10-a678-54706377d33e-kube-api-access-tsbr5\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.997990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.998000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-wtmp\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.998415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.998045    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-root\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.998415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.998117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e97cb6e-3ac8-4e10-a678-54706377d33e-metrics-client-ca\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:25:59.998415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:25:59.998183    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-sys\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-root\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e97cb6e-3ac8-4e10-a678-54706377d33e-metrics-client-ca\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099671    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-sys\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099735    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099756    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-root\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-textfile\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099831    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099865    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-tls\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsbr5\" (UniqueName: \"kubernetes.io/projected/9e97cb6e-3ac8-4e10-a678-54706377d33e-kube-api-access-tsbr5\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.099947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-wtmp\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.100024    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-sys\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.100077    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-textfile\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.100697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.100110    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-wtmp\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.101457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.100761    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.101457 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.100960    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e97cb6e-3ac8-4e10-a678-54706377d33e-metrics-client-ca\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.111133 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.111058    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-tls\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.111296 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.111255    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsbr5\" (UniqueName: \"kubernetes.io/projected/9e97cb6e-3ac8-4e10-a678-54706377d33e-kube-api-access-tsbr5\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.111562 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.111525    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e97cb6e-3ac8-4e10-a678-54706377d33e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkxnv\" (UID: \"9e97cb6e-3ac8-4e10-a678-54706377d33e\") " pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.247557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.247515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rkxnv"
Jul 09 12:26:00.261080 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:26:00.261052    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e97cb6e_3ac8_4e10_a678_54706377d33e.slice/crio-4ea706ed9172be02bbc24d36bcf6eb9436c1e4a03291566d32d86a90e88fd98e WatchSource:0}: Error finding container 4ea706ed9172be02bbc24d36bcf6eb9436c1e4a03291566d32d86a90e88fd98e: Status 404 returned error can't find the container with id 4ea706ed9172be02bbc24d36bcf6eb9436c1e4a03291566d32d86a90e88fd98e
Jul 09 12:26:00.480581 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:00.480538    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkxnv" event={"ID":"9e97cb6e-3ac8-4e10-a678-54706377d33e","Type":"ContainerStarted","Data":"4ea706ed9172be02bbc24d36bcf6eb9436c1e4a03291566d32d86a90e88fd98e"}
Jul 09 12:26:01.486896 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:01.486855    2574 generic.go:356] "Generic (PLEG): container finished" podID="9e97cb6e-3ac8-4e10-a678-54706377d33e" containerID="613a6262ee74cebed9905835ecf7975d7540b1e437253c0543f9029145765f77" exitCode=0
Jul 09 12:26:01.487347 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:01.486923    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkxnv" event={"ID":"9e97cb6e-3ac8-4e10-a678-54706377d33e","Type":"ContainerDied","Data":"613a6262ee74cebed9905835ecf7975d7540b1e437253c0543f9029145765f77"}
Jul 09 12:26:02.494360 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:02.494329    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkxnv" event={"ID":"9e97cb6e-3ac8-4e10-a678-54706377d33e","Type":"ContainerStarted","Data":"4c575347cc7fd562f15c8e97486b44d833120874f396efbf3e1ddd348159c52c"}
Jul 09 12:26:02.494751 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:02.494380    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkxnv" event={"ID":"9e97cb6e-3ac8-4e10-a678-54706377d33e","Type":"ContainerStarted","Data":"250da6afa2cf594ace2ce06d6a92a932f73f74f1e1e71ce7ac9368a46c821b72"}
Jul 09 12:26:12.008303 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:12.008228    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rkxnv" podStartSLOduration=11.877373582 podStartE2EDuration="13.008209892s" totalImagesPullingTime="1.05862669s" totalInitContainerRuntime="72.20962ms" isStatefulPod=true podCreationTimestamp="2026-07-09 12:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:26:02.529776866 +0000 UTC m=+120.124400910" watchObservedRunningTime="2026-07-09 12:26:12.008209892 +0000 UTC m=+129.602833935"
Jul 09 12:26:12.008940 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:12.008914    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644c647d7c-8c9gs"]
Jul 09 12:26:12.724255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:12.724207    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:26:12.726883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:12.726846    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab0cfc02-fc44-4800-8624-37e26ac4f333-metrics-certs\") pod \"network-metrics-daemon-l72qj\" (UID: \"ab0cfc02-fc44-4800-8624-37e26ac4f333\") " pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:26:12.790941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:12.790908    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x4nrr\""
Jul 09 12:26:12.798334 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:12.798302    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l72qj"
Jul 09 12:26:13.058900 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.058787    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dfbd7d9cf-h87fm"]
Jul 09 12:26:13.062301 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.062271    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.067093 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.067065    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 12:26:13.067937 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.067913    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 12:26:13.068222 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.068206    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 12:26:13.068554 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.068533    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 12:26:13.068652 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.068625    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 12:26:13.069136 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.069118    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nj5qb\""
Jul 09 12:26:13.074501 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.074472    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 12:26:13.083443 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.083413    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfbd7d9cf-h87fm"]
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-oauth-config\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-trusted-ca-bundle\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127552    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-serving-cert\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-oauth-serving-cert\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127646    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-config\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-service-ca\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.127829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.127726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kstjg\" (UniqueName: \"kubernetes.io/projected/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-kube-api-access-kstjg\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.226565 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.226544    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l72qj"]
Jul 09 12:26:13.228380 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-oauth-serving-cert\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.228527 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-config\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.228527 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-service-ca\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.228527 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kstjg\" (UniqueName: \"kubernetes.io/projected/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-kube-api-access-kstjg\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.228527 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-oauth-config\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.228746 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-trusted-ca-bundle\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.228746 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.228568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-serving-cert\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.229347 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.229199    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-oauth-serving-cert\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.229347 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.229219    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-config\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.229506 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:26:13.229383    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0cfc02_fc44_4800_8624_37e26ac4f333.slice/crio-688716351dc250274fa1b63ce5e6b34da559faf26465d70420a6b98d45f69361 WatchSource:0}: Error finding container 688716351dc250274fa1b63ce5e6b34da559faf26465d70420a6b98d45f69361: Status 404 returned error can't find the container with id 688716351dc250274fa1b63ce5e6b34da559faf26465d70420a6b98d45f69361
Jul 09 12:26:13.229615 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.229589    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-service-ca\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.229801 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.229773    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-trusted-ca-bundle\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.231443 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.231424    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-oauth-config\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.231551 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.231532    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-serving-cert\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.254580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.254555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kstjg\" (UniqueName: \"kubernetes.io/projected/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-kube-api-access-kstjg\") pod \"console-5dfbd7d9cf-h87fm\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") " pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.378937 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.378857    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:13.536667 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.536591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfbd7d9cf-h87fm"]
Jul 09 12:26:13.540664 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.540090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-fgmgf" event={"ID":"92c091f5-c366-46fe-9012-9a05a80a2898","Type":"ContainerStarted","Data":"0980f62171e0be7feed1b715c327d496e972e480bcb24ddd408c56099e5c572a"}
Jul 09 12:26:13.540664 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:26:13.540211    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876c18e4_18cf_4a0e_a8b6_dbd795af5b2f.slice/crio-670af09219e9c687d0cacda252da718bc53eaaa72ceb5d721d0666755df8fa33 WatchSource:0}: Error finding container 670af09219e9c687d0cacda252da718bc53eaaa72ceb5d721d0666755df8fa33: Status 404 returned error can't find the container with id 670af09219e9c687d0cacda252da718bc53eaaa72ceb5d721d0666755df8fa33
Jul 09 12:26:13.540664 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.540475    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:26:13.542254 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.542092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l72qj" event={"ID":"ab0cfc02-fc44-4800-8624-37e26ac4f333","Type":"ContainerStarted","Data":"688716351dc250274fa1b63ce5e6b34da559faf26465d70420a6b98d45f69361"}
Jul 09 12:26:13.551265 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.551240    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-fgmgf"
Jul 09 12:26:13.568396 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:13.568343    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-fgmgf" podStartSLOduration=1.88109934 podStartE2EDuration="20.568326063s" totalImagesPullingTime="18.687226723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:26:13.566735436 +0000 UTC m=+131.161359480" watchObservedRunningTime="2026-07-09 12:26:13.568326063 +0000 UTC m=+131.162950106"
Jul 09 12:26:14.552288 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:14.552230    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfbd7d9cf-h87fm" event={"ID":"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f","Type":"ContainerStarted","Data":"670af09219e9c687d0cacda252da718bc53eaaa72ceb5d721d0666755df8fa33"}
Jul 09 12:26:14.949929 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:14.949885    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" containerID="cri-o://7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512" gracePeriod=30
Jul 09 12:26:15.559895 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.559759    2574 generic.go:356] "Generic (PLEG): container finished" podID="d5395c69-4eb4-433d-a30b-387bcb6b2d8f" containerID="e9259376c223625fe5aa1910c3927617c236fb8c358628154e080b21ffb2ba13" exitCode=0
Jul 09 12:26:15.559895 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.559801    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb" event={"ID":"d5395c69-4eb4-433d-a30b-387bcb6b2d8f","Type":"ContainerDied","Data":"e9259376c223625fe5aa1910c3927617c236fb8c358628154e080b21ffb2ba13"}
Jul 09 12:26:15.560701 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.560661    2574 scope.go:117] "RemoveContainer" containerID="e9259376c223625fe5aa1910c3927617c236fb8c358628154e080b21ffb2ba13"
Jul 09 12:26:15.563988 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.563958    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l72qj" event={"ID":"ab0cfc02-fc44-4800-8624-37e26ac4f333","Type":"ContainerStarted","Data":"91582cd2adfa82ab0fe90fb46bf37721342963ac0c3aa91ad8f9efce3e1936c4"}
Jul 09 12:26:15.564115 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.564021    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l72qj" event={"ID":"ab0cfc02-fc44-4800-8624-37e26ac4f333","Type":"ContainerStarted","Data":"2062a47e33d7d20c8fa1e6e9e8528ca1d9596439be3657c29400a4df4a2ae2e4"}
Jul 09 12:26:15.567317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.567197    2574 generic.go:356] "Generic (PLEG): container finished" podID="2cba3a95-1272-4252-ac50-51db912be7a9" containerID="7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512" exitCode=0
Jul 09 12:26:15.567317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.567245    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" event={"ID":"2cba3a95-1272-4252-ac50-51db912be7a9","Type":"ContainerDied","Data":"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"}
Jul 09 12:26:15.567317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.567282    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" event={"ID":"2cba3a95-1272-4252-ac50-51db912be7a9","Type":"ContainerStarted","Data":"f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766"}
Jul 09 12:26:15.567933 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.567842    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:26:15.733151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:15.733088    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l72qj" podStartSLOduration=131.322623934 podStartE2EDuration="2m12.733066773s" totalImagesPullingTime="1.410442839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:26:15.73237056 +0000 UTC m=+133.326994615" watchObservedRunningTime="2026-07-09 12:26:15.733066773 +0000 UTC m=+133.327690815"
Jul 09 12:26:16.577266 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:16.577192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jn6pb" event={"ID":"d5395c69-4eb4-433d-a30b-387bcb6b2d8f","Type":"ContainerStarted","Data":"670ab3f63e29bc58223d63e7ed469b83427742fc825bfc4c209ffc4e4e6408af"}
Jul 09 12:26:18.586729 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:18.586674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfbd7d9cf-h87fm" event={"ID":"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f","Type":"ContainerStarted","Data":"9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb"}
Jul 09 12:26:18.624349 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:18.624271    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dfbd7d9cf-h87fm" podStartSLOduration=1.5438052610000002 podStartE2EDuration="5.62424803s" totalImagesPullingTime="4.080442769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:26:18.623403337 +0000 UTC m=+136.218027381" watchObservedRunningTime="2026-07-09 12:26:18.62424803 +0000 UTC m=+136.218872077"
Jul 09 12:26:23.379497 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:23.379452    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:23.380018 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:23.379521    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:23.384398 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:23.384373    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:23.616099 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:23.616066    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:26:35.572952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:35.572920    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:26:40.599382 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.599331    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry" containerID="cri-o://f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766" gracePeriod=30
Jul 09 12:26:40.843550 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.843525    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:26:40.963122 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963096    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2cba3a95-1272-4252-ac50-51db912be7a9-ca-trust-extracted\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963135    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-bound-sa-token\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963164    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-image-registry-private-configuration\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963230    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963258    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-installation-pull-secrets\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963286    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-registry-certificates\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963556 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963321    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmtlq\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-kube-api-access-gmtlq\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.963556 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963352    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-trusted-ca\") pod \"2cba3a95-1272-4252-ac50-51db912be7a9\" (UID: \"2cba3a95-1272-4252-ac50-51db912be7a9\") "
Jul 09 12:26:40.964032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.963995    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:26:40.964153 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.964004    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:26:40.965902 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.965870    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:26:40.966020 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.965984    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:26:40.966162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.966137    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:26:40.966544 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.966522    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:26:40.966612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.966518    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-kube-api-access-gmtlq" (OuterVolumeSpecName: "kube-api-access-gmtlq") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "kube-api-access-gmtlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:26:40.972802 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:40.972774    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cba3a95-1272-4252-ac50-51db912be7a9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2cba3a95-1272-4252-ac50-51db912be7a9" (UID: "2cba3a95-1272-4252-ac50-51db912be7a9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:26:41.064904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064864    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmtlq\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-kube-api-access-gmtlq\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.064904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064896    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-trusted-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.064904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064905    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2cba3a95-1272-4252-ac50-51db912be7a9-ca-trust-extracted\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.064904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064913    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-bound-sa-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.065236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064922    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-image-registry-private-configuration\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.065236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064930    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2cba3a95-1272-4252-ac50-51db912be7a9-registry-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.065236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064941    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2cba3a95-1272-4252-ac50-51db912be7a9-installation-pull-secrets\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.065236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.064948    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2cba3a95-1272-4252-ac50-51db912be7a9-registry-certificates\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:26:41.681822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.681729    2574 generic.go:356] "Generic (PLEG): container finished" podID="2cba3a95-1272-4252-ac50-51db912be7a9" containerID="f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766" exitCode=0
Jul 09 12:26:41.681822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.681780    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs"
Jul 09 12:26:41.682261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.681816    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" event={"ID":"2cba3a95-1272-4252-ac50-51db912be7a9","Type":"ContainerDied","Data":"f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766"}
Jul 09 12:26:41.682261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.681869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644c647d7c-8c9gs" event={"ID":"2cba3a95-1272-4252-ac50-51db912be7a9","Type":"ContainerDied","Data":"04e4d049bc8bde99e70f5f0de8d6f3199d1dfb005d59dcb461f3b6d859ca1c8f"}
Jul 09 12:26:41.682261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.681895    2574 scope.go:117] "RemoveContainer" containerID="f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766"
Jul 09 12:26:41.690157 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.690135    2574 scope.go:117] "RemoveContainer" containerID="7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"
Jul 09 12:26:41.697492 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.697472    2574 scope.go:117] "RemoveContainer" containerID="f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766"
Jul 09 12:26:41.697753 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:26:41.697732    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766\": container with ID starting with f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766 not found: ID does not exist" containerID="f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766"
Jul 09 12:26:41.697824 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.697762    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766"} err="failed to get container status \"f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766\": rpc error: code = NotFound desc = could not find container \"f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766\": container with ID starting with f82344d94b8e6c47e3744b29021217deb3047703e87f90850f77bb9c46349766 not found: ID does not exist"
Jul 09 12:26:41.697824 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.697779    2574 scope.go:117] "RemoveContainer" containerID="7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"
Jul 09 12:26:41.698026 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:26:41.698011    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512\": container with ID starting with 7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512 not found: ID does not exist" containerID="7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"
Jul 09 12:26:41.698071 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.698028    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512"} err="failed to get container status \"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512\": rpc error: code = NotFound desc = could not find container \"7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512\": container with ID starting with 7a7b3b15fe9973d5a9402298ae44a17f1d417773fe56393cf91316718b931512 not found: ID does not exist"
Jul 09 12:26:41.707418 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.707395    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644c647d7c-8c9gs"]
Jul 09 12:26:41.712082 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:41.712062    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-644c647d7c-8c9gs"]
Jul 09 12:26:42.969563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:26:42.969532    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" path="/var/lib/kubelet/pods/2cba3a95-1272-4252-ac50-51db912be7a9/volumes"
Jul 09 12:27:28.409806 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:28.409763    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfbd7d9cf-h87fm"]
Jul 09 12:27:53.430265 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.430220    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5dfbd7d9cf-h87fm" podUID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" containerName="console" containerID="cri-o://9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb" gracePeriod=15
Jul 09 12:27:53.673284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.673261    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:27:53.771618 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.771582    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-oauth-serving-cert\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.771861 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.771629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-trusted-ca-bundle\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.771861 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.771662    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-serving-cert\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.771861 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.771834    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-config\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.772043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.771882    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kstjg\" (UniqueName: \"kubernetes.io/projected/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-kube-api-access-kstjg\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.772043 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.771927    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-service-ca\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.772144 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772117    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-oauth-config\") pod \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\" (UID: \"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f\") "
Jul 09 12:27:53.772195 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772156    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:27:53.772246 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772187    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-config" (OuterVolumeSpecName: "console-config") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:27:53.772246 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772200    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:27:53.772329 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772262    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-service-ca" (OuterVolumeSpecName: "service-ca") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:27:53.772390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772373    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-oauth-serving-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.772427 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772397    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-trusted-ca-bundle\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.772427 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772412    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-config\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.772427 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.772424    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.774053 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.774032    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:27:53.774119 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.774049    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:27:53.774176 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.774158    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-kube-api-access-kstjg" (OuterVolumeSpecName: "kube-api-access-kstjg") pod "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" (UID: "876c18e4-18cf-4a0e-a8b6-dbd795af5b2f"). InnerVolumeSpecName "kube-api-access-kstjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:27:53.873302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.873260    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-oauth-config\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.873302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.873294    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-console-serving-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.873302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.873304    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kstjg\" (UniqueName: \"kubernetes.io/projected/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f-kube-api-access-kstjg\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:27:53.942747 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.942708    2574 generic.go:356] "Generic (PLEG): container finished" podID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" containerID="9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb" exitCode=0
Jul 09 12:27:53.942747 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.942751    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfbd7d9cf-h87fm"
Jul 09 12:27:53.942941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.942759    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfbd7d9cf-h87fm" event={"ID":"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f","Type":"ContainerDied","Data":"9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb"}
Jul 09 12:27:53.942941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.942810    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfbd7d9cf-h87fm" event={"ID":"876c18e4-18cf-4a0e-a8b6-dbd795af5b2f","Type":"ContainerDied","Data":"670af09219e9c687d0cacda252da718bc53eaaa72ceb5d721d0666755df8fa33"}
Jul 09 12:27:53.942941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.942836    2574 scope.go:117] "RemoveContainer" containerID="9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb"
Jul 09 12:27:53.950937 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.950919    2574 scope.go:117] "RemoveContainer" containerID="9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb"
Jul 09 12:27:53.951200 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:27:53.951181    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb\": container with ID starting with 9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb not found: ID does not exist" containerID="9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb"
Jul 09 12:27:53.951249 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.951211    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb"} err="failed to get container status \"9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb\": rpc error: code = NotFound desc = could not find container \"9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb\": container with ID starting with 9fd8e6de325c926797a2ce6a6643018b1439d85c60e163da82735d62a52dabdb not found: ID does not exist"
Jul 09 12:27:53.964178 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.964150    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfbd7d9cf-h87fm"]
Jul 09 12:27:53.966143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:53.966122    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dfbd7d9cf-h87fm"]
Jul 09 12:27:54.613160 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:54.613117    2574 patch_prober.go:28] interesting pod/console-5dfbd7d9cf-h87fm container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.18:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 09 12:27:54.613568 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:54.613189    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-5dfbd7d9cf-h87fm" podUID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 09 12:27:54.969787 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:27:54.969754    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" path="/var/lib/kubelet/pods/876c18e4-18cf-4a0e-a8b6-dbd795af5b2f/volumes"
Jul 09 12:28:56.228866 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.228831    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"]
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229092    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229104    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229120    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" containerName="console"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229125    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" containerName="console"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229134    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229140    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229195    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry"
Jul 09 12:28:56.229317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.229203    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="876c18e4-18cf-4a0e-a8b6-dbd795af5b2f" containerName="console"
Jul 09 12:28:56.230876 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.230861    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.235910 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.235880    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5v55s\""
Jul 09 12:28:56.236063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.235979    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 09 12:28:56.236063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.235985    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:28:56.259928 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.259895    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"]
Jul 09 12:28:56.314607 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.314568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14de1a40-8bec-4cf1-bf15-7a2ddac78754-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mx7c\" (UID: \"14de1a40-8bec-4cf1-bf15-7a2ddac78754\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.314790 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.314661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z5pp\" (UniqueName: \"kubernetes.io/projected/14de1a40-8bec-4cf1-bf15-7a2ddac78754-kube-api-access-6z5pp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mx7c\" (UID: \"14de1a40-8bec-4cf1-bf15-7a2ddac78754\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.415172 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.415135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14de1a40-8bec-4cf1-bf15-7a2ddac78754-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mx7c\" (UID: \"14de1a40-8bec-4cf1-bf15-7a2ddac78754\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.415364 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.415229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z5pp\" (UniqueName: \"kubernetes.io/projected/14de1a40-8bec-4cf1-bf15-7a2ddac78754-kube-api-access-6z5pp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mx7c\" (UID: \"14de1a40-8bec-4cf1-bf15-7a2ddac78754\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.415531 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.415512    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14de1a40-8bec-4cf1-bf15-7a2ddac78754-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mx7c\" (UID: \"14de1a40-8bec-4cf1-bf15-7a2ddac78754\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.424541 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.424508    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z5pp\" (UniqueName: \"kubernetes.io/projected/14de1a40-8bec-4cf1-bf15-7a2ddac78754-kube-api-access-6z5pp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7mx7c\" (UID: \"14de1a40-8bec-4cf1-bf15-7a2ddac78754\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.539361 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.539266    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"
Jul 09 12:28:56.682711 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:56.682690    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c"]
Jul 09 12:28:56.685659 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:28:56.685630    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14de1a40_8bec_4cf1_bf15_7a2ddac78754.slice/crio-aeb780cdc43cde66a20a40fb0f2135d4b5f6f9cd9962440d077fdda2dab32269 WatchSource:0}: Error finding container aeb780cdc43cde66a20a40fb0f2135d4b5f6f9cd9962440d077fdda2dab32269: Status 404 returned error can't find the container with id aeb780cdc43cde66a20a40fb0f2135d4b5f6f9cd9962440d077fdda2dab32269
Jul 09 12:28:57.165011 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:28:57.164972    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c" event={"ID":"14de1a40-8bec-4cf1-bf15-7a2ddac78754","Type":"ContainerStarted","Data":"aeb780cdc43cde66a20a40fb0f2135d4b5f6f9cd9962440d077fdda2dab32269"}
Jul 09 12:29:00.181159 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:00.181124    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c" event={"ID":"14de1a40-8bec-4cf1-bf15-7a2ddac78754","Type":"ContainerStarted","Data":"b08e9918649445b1eebc42c1ae5675aa6704df9475a35de45ee0b7b4c995460e"}
Jul 09 12:29:00.229187 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:00.229132    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7mx7c" podStartSLOduration=1.77782822 podStartE2EDuration="4.229115136s" totalImagesPullingTime="2.451286916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:29:00.226366818 +0000 UTC m=+297.820990879" watchObservedRunningTime="2026-07-09 12:29:00.229115136 +0000 UTC m=+297.823739179"
Jul 09 12:29:02.848022 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:02.847994    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:29:02.858441 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:02.858412    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:29:02.859535 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:02.859514    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 12:29:12.052750 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.052716    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h52p7"]
Jul 09 12:29:12.057729 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.053012    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cba3a95-1272-4252-ac50-51db912be7a9" containerName="registry"
Jul 09 12:29:12.058808 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.058789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.062546 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.062513    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 12:29:12.062711 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.062641    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-x5hw7\""
Jul 09 12:29:12.063520 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.063505    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 12:29:12.072554 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.072523    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h52p7"]
Jul 09 12:29:12.148400 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.148365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5289169-bbd7-468a-ba39-e90d734e61b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-h52p7\" (UID: \"d5289169-bbd7-468a-ba39-e90d734e61b5\") " pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.148583 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.148408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q895m\" (UniqueName: \"kubernetes.io/projected/d5289169-bbd7-468a-ba39-e90d734e61b5-kube-api-access-q895m\") pod \"cert-manager-55b4d954d5-h52p7\" (UID: \"d5289169-bbd7-468a-ba39-e90d734e61b5\") " pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.249163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.249115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5289169-bbd7-468a-ba39-e90d734e61b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-h52p7\" (UID: \"d5289169-bbd7-468a-ba39-e90d734e61b5\") " pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.249347 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.249175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q895m\" (UniqueName: \"kubernetes.io/projected/d5289169-bbd7-468a-ba39-e90d734e61b5-kube-api-access-q895m\") pod \"cert-manager-55b4d954d5-h52p7\" (UID: \"d5289169-bbd7-468a-ba39-e90d734e61b5\") " pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.259199 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.259167    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q895m\" (UniqueName: \"kubernetes.io/projected/d5289169-bbd7-468a-ba39-e90d734e61b5-kube-api-access-q895m\") pod \"cert-manager-55b4d954d5-h52p7\" (UID: \"d5289169-bbd7-468a-ba39-e90d734e61b5\") " pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.260588 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.260568    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5289169-bbd7-468a-ba39-e90d734e61b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-h52p7\" (UID: \"d5289169-bbd7-468a-ba39-e90d734e61b5\") " pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.367984 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.367893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h52p7"
Jul 09 12:29:12.513885 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.513854    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h52p7"]
Jul 09 12:29:12.517576 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:29:12.517548    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5289169_bbd7_468a_ba39_e90d734e61b5.slice/crio-a97c4ac70860f27ffa2cb67efa965abf5f56812e2b5984aec8e04a685a1749bb WatchSource:0}: Error finding container a97c4ac70860f27ffa2cb67efa965abf5f56812e2b5984aec8e04a685a1749bb: Status 404 returned error can't find the container with id a97c4ac70860f27ffa2cb67efa965abf5f56812e2b5984aec8e04a685a1749bb
Jul 09 12:29:12.519899 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:12.519882    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:29:13.233305 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.233268    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h52p7" event={"ID":"d5289169-bbd7-468a-ba39-e90d734e61b5","Type":"ContainerStarted","Data":"a97c4ac70860f27ffa2cb67efa965abf5f56812e2b5984aec8e04a685a1749bb"}
Jul 09 12:29:13.312755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.312712    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"]
Jul 09 12:29:13.315096 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.315076    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.319115 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.319095    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:29:13.319255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.319137    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 12:29:13.320077 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.320050    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4c44r\""
Jul 09 12:29:13.339914 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.339882    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"]
Jul 09 12:29:13.358495 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.358460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9afa0d2-94fa-47be-81f6-3ff93f5ff182-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jzfrq\" (UID: \"d9afa0d2-94fa-47be-81f6-3ff93f5ff182\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.358756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.358506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfj8g\" (UniqueName: \"kubernetes.io/projected/d9afa0d2-94fa-47be-81f6-3ff93f5ff182-kube-api-access-vfj8g\") pod \"openshift-lws-operator-fd8ccff4c-jzfrq\" (UID: \"d9afa0d2-94fa-47be-81f6-3ff93f5ff182\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.459810 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.459772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9afa0d2-94fa-47be-81f6-3ff93f5ff182-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jzfrq\" (UID: \"d9afa0d2-94fa-47be-81f6-3ff93f5ff182\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.459810 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.459824    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfj8g\" (UniqueName: \"kubernetes.io/projected/d9afa0d2-94fa-47be-81f6-3ff93f5ff182-kube-api-access-vfj8g\") pod \"openshift-lws-operator-fd8ccff4c-jzfrq\" (UID: \"d9afa0d2-94fa-47be-81f6-3ff93f5ff182\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.460237 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.460215    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9afa0d2-94fa-47be-81f6-3ff93f5ff182-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jzfrq\" (UID: \"d9afa0d2-94fa-47be-81f6-3ff93f5ff182\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.474218 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.474185    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfj8g\" (UniqueName: \"kubernetes.io/projected/d9afa0d2-94fa-47be-81f6-3ff93f5ff182-kube-api-access-vfj8g\") pod \"openshift-lws-operator-fd8ccff4c-jzfrq\" (UID: \"d9afa0d2-94fa-47be-81f6-3ff93f5ff182\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.626301 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.626224    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"
Jul 09 12:29:13.788591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:13.788555    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq"]
Jul 09 12:29:13.793601 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:29:13.793563    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9afa0d2_94fa_47be_81f6_3ff93f5ff182.slice/crio-2df0f12cf5cc46cf9c30631f4c31645b514aa7191c4ba2b756b2e780a2951d3a WatchSource:0}: Error finding container 2df0f12cf5cc46cf9c30631f4c31645b514aa7191c4ba2b756b2e780a2951d3a: Status 404 returned error can't find the container with id 2df0f12cf5cc46cf9c30631f4c31645b514aa7191c4ba2b756b2e780a2951d3a
Jul 09 12:29:14.239387 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:14.239346    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq" event={"ID":"d9afa0d2-94fa-47be-81f6-3ff93f5ff182","Type":"ContainerStarted","Data":"2df0f12cf5cc46cf9c30631f4c31645b514aa7191c4ba2b756b2e780a2951d3a"}
Jul 09 12:29:15.246356 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:15.246296    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h52p7" event={"ID":"d5289169-bbd7-468a-ba39-e90d734e61b5","Type":"ContainerStarted","Data":"1b93aba31db788518957e00d6c11860338c79d600f614ee4d274cda3665a78da"}
Jul 09 12:29:15.274253 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:15.274193    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-h52p7" podStartSLOduration=0.944414847 podStartE2EDuration="3.274171852s" totalImagesPullingTime="2.329757005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:29:15.271480317 +0000 UTC m=+312.866104384" watchObservedRunningTime="2026-07-09 12:29:15.274171852 +0000 UTC m=+312.868795896"
Jul 09 12:29:17.256310 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:17.256269    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq" event={"ID":"d9afa0d2-94fa-47be-81f6-3ff93f5ff182","Type":"ContainerStarted","Data":"92b11d553089903cac5be5dd2a4af57ab11a14b7ba96b70a444bb7f0d8e68b75"}
Jul 09 12:29:33.885261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.885203    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jzfrq" podStartSLOduration=18.002799843 podStartE2EDuration="20.885175213s" totalImagesPullingTime="2.88237537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:29:17.284277995 +0000 UTC m=+314.878902040" watchObservedRunningTime="2026-07-09 12:29:33.885175213 +0000 UTC m=+331.479799256"
Jul 09 12:29:33.885812 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.885795    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"]
Jul 09 12:29:33.888811 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.888791    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:33.910814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.910786    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 12:29:33.926284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.926257    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 12:29:33.927580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.927562    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2gqgj\""
Jul 09 12:29:33.929716 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.929675    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 12:29:33.933216 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.933201    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 12:29:33.962475 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:33.962441    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"]
Jul 09 12:29:34.025310 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.025274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e655918b-ac7a-4289-8251-15a6a68078ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.025460 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.025320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpcx9\" (UniqueName: \"kubernetes.io/projected/e655918b-ac7a-4289-8251-15a6a68078ce-kube-api-access-tpcx9\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.025460 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.025344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e655918b-ac7a-4289-8251-15a6a68078ce-manifests\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.025460 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.025362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e655918b-ac7a-4289-8251-15a6a68078ce-tmp\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.025460 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.025407    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e655918b-ac7a-4289-8251-15a6a68078ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.126189 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.126149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpcx9\" (UniqueName: \"kubernetes.io/projected/e655918b-ac7a-4289-8251-15a6a68078ce-kube-api-access-tpcx9\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.126403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.126208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e655918b-ac7a-4289-8251-15a6a68078ce-manifests\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.126403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.126235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e655918b-ac7a-4289-8251-15a6a68078ce-tmp\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.126403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.126298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e655918b-ac7a-4289-8251-15a6a68078ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.126403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.126345    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e655918b-ac7a-4289-8251-15a6a68078ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.126766 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.126726    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e655918b-ac7a-4289-8251-15a6a68078ce-manifests\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.128702 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.128660    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e655918b-ac7a-4289-8251-15a6a68078ce-tmp\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.128842 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.128824    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e655918b-ac7a-4289-8251-15a6a68078ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.129007 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.128986    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e655918b-ac7a-4289-8251-15a6a68078ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.152082 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.152014    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpcx9\" (UniqueName: \"kubernetes.io/projected/e655918b-ac7a-4289-8251-15a6a68078ce-kube-api-access-tpcx9\") pod \"opendatahub-operator-controller-manager-578cb9c586-vhgg5\" (UID: \"e655918b-ac7a-4289-8251-15a6a68078ce\") " pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.199007 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.198965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:34.371731 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:34.371671    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"]
Jul 09 12:29:34.374108 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:29:34.374077    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode655918b_ac7a_4289_8251_15a6a68078ce.slice/crio-8e366c569706cd75cf90c93a13adc989d206c30a0595608e638f94ad62ea0a3a WatchSource:0}: Error finding container 8e366c569706cd75cf90c93a13adc989d206c30a0595608e638f94ad62ea0a3a: Status 404 returned error can't find the container with id 8e366c569706cd75cf90c93a13adc989d206c30a0595608e638f94ad62ea0a3a
Jul 09 12:29:35.327927 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:35.327894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5" event={"ID":"e655918b-ac7a-4289-8251-15a6a68078ce","Type":"ContainerStarted","Data":"8e366c569706cd75cf90c93a13adc989d206c30a0595608e638f94ad62ea0a3a"}
Jul 09 12:29:38.343886 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:38.343848    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5" event={"ID":"e655918b-ac7a-4289-8251-15a6a68078ce","Type":"ContainerStarted","Data":"e034d3d154af51c7794e5e9bf75074c9e1960c06bc7053e63832ce9462e51929"}
Jul 09 12:29:39.348815 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:39.348775    2574 generic.go:356] "Generic (PLEG): container finished" podID="e655918b-ac7a-4289-8251-15a6a68078ce" containerID="e034d3d154af51c7794e5e9bf75074c9e1960c06bc7053e63832ce9462e51929" exitCode=0
Jul 09 12:29:39.349198 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:39.348827    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5" event={"ID":"e655918b-ac7a-4289-8251-15a6a68078ce","Type":"ContainerDied","Data":"e034d3d154af51c7794e5e9bf75074c9e1960c06bc7053e63832ce9462e51929"}
Jul 09 12:29:40.355290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:40.355250    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5" event={"ID":"e655918b-ac7a-4289-8251-15a6a68078ce","Type":"ContainerStarted","Data":"0289227616d65a39819e292be242ebc5e4bacb11f1cc50b82cb12e123b91d5fb"}
Jul 09 12:29:40.355664 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:40.355366    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:40.387137 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:40.387077    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5" podStartSLOduration=2.730535663 podStartE2EDuration="7.387059697s" totalImagesPullingTime="3.571238669s" totalInitContainerRuntime="1.085285365s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:29:40.386779844 +0000 UTC m=+337.981403906" watchObservedRunningTime="2026-07-09 12:29:40.387059697 +0000 UTC m=+337.981683761"
Jul 09 12:29:51.361867 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:51.361832    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-578cb9c586-vhgg5"
Jul 09 12:29:55.820059 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.819974    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gpsh7"]
Jul 09 12:29:55.823332 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.823311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:55.831570 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.831549    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-fnchm\""
Jul 09 12:29:55.838452 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.838429    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 09 12:29:55.959604 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.959564    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7776979d9-blj2f"]
Jul 09 12:29:55.962853 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.962829    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:55.965767 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:55.965741    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-dockercfg-ltk2m\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ltk2m\"" type="*v1.Secret"
Jul 09 12:29:55.966016 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:55.965989    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-7776979d9-blj2f\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="ea98560d-f9c3-4522-b760-ddc91625dcf9" pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:55.966119 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:55.966092    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 09 12:29:55.966814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.966796    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 12:29:55.966911 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.966849    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 12:29:55.969906 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.969888    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 12:29:55.985450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.985423    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gpsh7"]
Jul 09 12:29:55.992532 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.992508    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgrf5\" (UniqueName: \"kubernetes.io/projected/90d9cc88-72fc-4dd3-969a-4680b2bafc44-kube-api-access-qgrf5\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:55.992653 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:55.992549    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:56.031370 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.031335    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7776979d9-blj2f"]
Jul 09 12:29:56.093079 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.092991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksczt\" (UniqueName: \"kubernetes.io/projected/ea98560d-f9c3-4522-b760-ddc91625dcf9-kube-api-access-ksczt\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.093079 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.093045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrf5\" (UniqueName: \"kubernetes.io/projected/90d9cc88-72fc-4dd3-969a-4680b2bafc44-kube-api-access-qgrf5\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:56.093284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.093134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:56.093284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.093180    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea98560d-f9c3-4522-b760-ddc91625dcf9-tls-certs\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.093284 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:56.093214    2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 09 12:29:56.093284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.093220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea98560d-f9c3-4522-b760-ddc91625dcf9-tmp\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.093284 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:56.093279    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert podName:90d9cc88-72fc-4dd3-969a-4680b2bafc44 nodeName:}" failed. No retries permitted until 2026-07-09 12:29:56.593257635 +0000 UTC m=+354.187881656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert") pod "odh-model-controller-7998c67dc4-gpsh7" (UID: "90d9cc88-72fc-4dd3-969a-4680b2bafc44") : secret "odh-model-controller-webhook-cert" not found
Jul 09 12:29:56.153866 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.153834    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgrf5\" (UniqueName: \"kubernetes.io/projected/90d9cc88-72fc-4dd3-969a-4680b2bafc44-kube-api-access-qgrf5\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:56.193783 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.193747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea98560d-f9c3-4522-b760-ddc91625dcf9-tls-certs\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.193783 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.193787    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea98560d-f9c3-4522-b760-ddc91625dcf9-tmp\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.194011 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.193839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksczt\" (UniqueName: \"kubernetes.io/projected/ea98560d-f9c3-4522-b760-ddc91625dcf9-kube-api-access-ksczt\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.196183 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.196147    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea98560d-f9c3-4522-b760-ddc91625dcf9-tmp\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.196308 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.196276    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea98560d-f9c3-4522-b760-ddc91625dcf9-tls-certs\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:56.595949 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.595910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:56.596147 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:56.596069    2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 09 12:29:56.596202 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:29:56.596154    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert podName:90d9cc88-72fc-4dd3-969a-4680b2bafc44 nodeName:}" failed. No retries permitted until 2026-07-09 12:29:57.596133835 +0000 UTC m=+355.190757858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert") pod "odh-model-controller-7998c67dc4-gpsh7" (UID: "90d9cc88-72fc-4dd3-969a-4680b2bafc44") : secret "odh-model-controller-webhook-cert" not found
Jul 09 12:29:56.985962 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.985929    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 12:29:56.998861 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:56.998833    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksczt\" (UniqueName: \"kubernetes.io/projected/ea98560d-f9c3-4522-b760-ddc91625dcf9-kube-api-access-ksczt\") pod \"kube-auth-proxy-7776979d9-blj2f\" (UID: \"ea98560d-f9c3-4522-b760-ddc91625dcf9\") " pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:57.192757 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.192726    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ltk2m\""
Jul 09 12:29:57.202671 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.202641    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f"
Jul 09 12:29:57.436124 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:29:57.436097    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea98560d_f9c3_4522_b760_ddc91625dcf9.slice/crio-1d59d5af2c0cadb4875e7fe59a2f26e230ebcc77038cfeac89b78ca22bd9c546 WatchSource:0}: Error finding container 1d59d5af2c0cadb4875e7fe59a2f26e230ebcc77038cfeac89b78ca22bd9c546: Status 404 returned error can't find the container with id 1d59d5af2c0cadb4875e7fe59a2f26e230ebcc77038cfeac89b78ca22bd9c546
Jul 09 12:29:57.440803 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.440778    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7776979d9-blj2f"]
Jul 09 12:29:57.603296 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.603208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:57.605574 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.605552    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d9cc88-72fc-4dd3-969a-4680b2bafc44-cert\") pod \"odh-model-controller-7998c67dc4-gpsh7\" (UID: \"90d9cc88-72fc-4dd3-969a-4680b2bafc44\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:57.633755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.633707    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:29:57.783045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:57.783022    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gpsh7"]
Jul 09 12:29:57.784307 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:29:57.784280    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d9cc88_72fc_4dd3_969a_4680b2bafc44.slice/crio-a7e1a5cb7b9c917801a855085ce6c5d8f7ae8b23a71618a9d51ce4cc128a018e WatchSource:0}: Error finding container a7e1a5cb7b9c917801a855085ce6c5d8f7ae8b23a71618a9d51ce4cc128a018e: Status 404 returned error can't find the container with id a7e1a5cb7b9c917801a855085ce6c5d8f7ae8b23a71618a9d51ce4cc128a018e
Jul 09 12:29:58.058348 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.058299    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8m6g5"]
Jul 09 12:29:58.061841 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.061813    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.065286 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.065257    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 09 12:29:58.090042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.090006    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8m6g5"]
Jul 09 12:29:58.215533 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.215488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrbtp\" (UniqueName: \"kubernetes.io/projected/2af48309-8c34-4793-81b6-161b6bc05543-kube-api-access-xrbtp\") pod \"model-serving-api-54d89bd5fc-8m6g5\" (UID: \"2af48309-8c34-4793-81b6-161b6bc05543\") " pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.215746 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.215545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2af48309-8c34-4793-81b6-161b6bc05543-tls-certs\") pod \"model-serving-api-54d89bd5fc-8m6g5\" (UID: \"2af48309-8c34-4793-81b6-161b6bc05543\") " pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.316489 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.316389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrbtp\" (UniqueName: \"kubernetes.io/projected/2af48309-8c34-4793-81b6-161b6bc05543-kube-api-access-xrbtp\") pod \"model-serving-api-54d89bd5fc-8m6g5\" (UID: \"2af48309-8c34-4793-81b6-161b6bc05543\") " pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.316489 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.316457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2af48309-8c34-4793-81b6-161b6bc05543-tls-certs\") pod \"model-serving-api-54d89bd5fc-8m6g5\" (UID: \"2af48309-8c34-4793-81b6-161b6bc05543\") " pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.319323 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.319291    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2af48309-8c34-4793-81b6-161b6bc05543-tls-certs\") pod \"model-serving-api-54d89bd5fc-8m6g5\" (UID: \"2af48309-8c34-4793-81b6-161b6bc05543\") " pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.352625 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.352583    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrbtp\" (UniqueName: \"kubernetes.io/projected/2af48309-8c34-4793-81b6-161b6bc05543-kube-api-access-xrbtp\") pod \"model-serving-api-54d89bd5fc-8m6g5\" (UID: \"2af48309-8c34-4793-81b6-161b6bc05543\") " pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.375419 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.375383    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:29:58.436986 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.436896    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7" event={"ID":"90d9cc88-72fc-4dd3-969a-4680b2bafc44","Type":"ContainerStarted","Data":"a7e1a5cb7b9c917801a855085ce6c5d8f7ae8b23a71618a9d51ce4cc128a018e"}
Jul 09 12:29:58.438419 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.438346    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f" event={"ID":"ea98560d-f9c3-4522-b760-ddc91625dcf9","Type":"ContainerStarted","Data":"1d59d5af2c0cadb4875e7fe59a2f26e230ebcc77038cfeac89b78ca22bd9c546"}
Jul 09 12:29:58.559381 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:58.559350    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8m6g5"]
Jul 09 12:29:58.564038 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:29:58.564003    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af48309_8c34_4793_81b6_161b6bc05543.slice/crio-5c47f27db72ec01312b425cec8447200f2d6c5f58adf93b5003fbcfff06e3532 WatchSource:0}: Error finding container 5c47f27db72ec01312b425cec8447200f2d6c5f58adf93b5003fbcfff06e3532: Status 404 returned error can't find the container with id 5c47f27db72ec01312b425cec8447200f2d6c5f58adf93b5003fbcfff06e3532
Jul 09 12:29:59.445725 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.445628    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5" event={"ID":"2af48309-8c34-4793-81b6-161b6bc05543","Type":"ContainerStarted","Data":"5c47f27db72ec01312b425cec8447200f2d6c5f58adf93b5003fbcfff06e3532"}
Jul 09 12:29:59.958427 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.958393    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"]
Jul 09 12:29:59.961792 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.961763    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:29:59.979336 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.979296    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 12:29:59.979490 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.979399    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 12:29:59.979737 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.979719    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mpnsb\""
Jul 09 12:29:59.979737 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.979731    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 12:29:59.979857 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:29:59.979819    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 12:30:00.037558 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.037523    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"]
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-local-certs\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132461    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132491    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdkcf\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-kube-api-access-pdkcf\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-cacerts\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132634    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.132879 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.132720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-token\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234027 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.233926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-token\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234208 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.234166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-local-certs\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.234210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.234261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234393 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.234323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdkcf\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-kube-api-access-pdkcf\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234393 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.234369    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-cacerts\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.234500 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.234394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.235164 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.235110    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.237041 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.237011    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-local-certs\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.237295 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.237273    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.237378 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.237319    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.237876 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.237849    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-cacerts\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.284513 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.284481    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-token\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.286489 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.286453    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdkcf\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-kube-api-access-pdkcf\") pod \"istiod-openshift-gateway-7574459656-4xpj8\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:00.575242 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:00.575159    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:01.769231 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:01.769186    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"]
Jul 09 12:30:01.772351 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:30:01.772324    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d3d7a2_07cc_4a5f_a996_aa6b2c2e72a3.slice/crio-46cccc66882257dcd6ce47d01a35a6dee3bda67d3e6fe7eba1dd9b5854a263c3 WatchSource:0}: Error finding container 46cccc66882257dcd6ce47d01a35a6dee3bda67d3e6fe7eba1dd9b5854a263c3: Status 404 returned error can't find the container with id 46cccc66882257dcd6ce47d01a35a6dee3bda67d3e6fe7eba1dd9b5854a263c3
Jul 09 12:30:02.461856 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.461819    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7" event={"ID":"90d9cc88-72fc-4dd3-969a-4680b2bafc44","Type":"ContainerStarted","Data":"f7246e20325d9a4dbaf909d6bd1cab38c2af33033bbb60f366363c1d2a582236"}
Jul 09 12:30:02.462055 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.461894    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:30:02.463327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.463298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f" event={"ID":"ea98560d-f9c3-4522-b760-ddc91625dcf9","Type":"ContainerStarted","Data":"8cc859fc2826d124c0d432885d7d2bb84b7c26f95b646f872710871a742847ad"}
Jul 09 12:30:02.464695 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.464664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5" event={"ID":"2af48309-8c34-4793-81b6-161b6bc05543","Type":"ContainerStarted","Data":"187f90f4d7c0f6271c8cc1d730dcaa6f29d7d3834c64cf5548c65da25b28d70f"}
Jul 09 12:30:02.464790 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.464720    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:30:02.465802 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.465785    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" event={"ID":"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3","Type":"ContainerStarted","Data":"46cccc66882257dcd6ce47d01a35a6dee3bda67d3e6fe7eba1dd9b5854a263c3"}
Jul 09 12:30:02.565655 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.565598    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7" podStartSLOduration=3.753923094 podStartE2EDuration="7.56558094s" totalImagesPullingTime="3.811657846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:02.518666788 +0000 UTC m=+360.113290834" watchObservedRunningTime="2026-07-09 12:30:02.56558094 +0000 UTC m=+360.160204987"
Jul 09 12:30:02.566388 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:02.566357    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7776979d9-blj2f" podStartSLOduration=3.409805509 podStartE2EDuration="7.566346448s" totalImagesPullingTime="4.156540939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:02.562165872 +0000 UTC m=+360.156789917" watchObservedRunningTime="2026-07-09 12:30:02.566346448 +0000 UTC m=+360.160970489"
Jul 09 12:30:06.837489 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:06.837448    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:30:06.837883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:06.837532    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:30:07.490920 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:07.490883    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" event={"ID":"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3","Type":"ContainerStarted","Data":"3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1"}
Jul 09 12:30:07.491130 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:07.491089    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:07.492514 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:07.492488    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4xpj8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 12:30:07.492631 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:07.492545    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:30:07.530787 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:07.530723    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5" podStartSLOduration=6.498690433 podStartE2EDuration="9.530705967s" totalImagesPullingTime="3.032015534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:02.718833259 +0000 UTC m=+360.313457315" watchObservedRunningTime="2026-07-09 12:30:07.530705967 +0000 UTC m=+365.125330009"
Jul 09 12:30:07.531235 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:07.531201    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" podStartSLOduration=3.468402664 podStartE2EDuration="8.531190167s" totalImagesPullingTime="5.062787503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:07.52944144 +0000 UTC m=+365.124065485" watchObservedRunningTime="2026-07-09 12:30:07.531190167 +0000 UTC m=+365.125814209"
Jul 09 12:30:08.496207 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:08.496171    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4xpj8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 12:30:08.496580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:08.496231    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:30:10.038291 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.038249    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"]
Jul 09 12:30:10.041844 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.041822    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.045317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.045290    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vhth7\""
Jul 09 12:30:10.085050 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.085014    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"]
Jul 09 12:30:10.120251 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120436 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120436 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120285    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120436 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120564 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120564 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2k7d\" (UniqueName: \"kubernetes.io/projected/3d473c20-cdc7-45e7-9fb8-b241ae844d28-kube-api-access-k2k7d\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120564 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120502    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120564 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120523    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.120739 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.120592    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.221932 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.221894    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222095 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.221949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222095 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.221970    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222095 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.221988    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222095 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222095 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222095 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222057    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2k7d\" (UniqueName: \"kubernetes.io/projected/3d473c20-cdc7-45e7-9fb8-b241ae844d28-kube-api-access-k2k7d\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222405 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222405 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222405 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222389    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222561 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222430    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222561 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222453    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222667 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222615    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.222928 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.222905    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.224497 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.224473    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.224585 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.224571    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.259749 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.259713    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d473c20-cdc7-45e7-9fb8-b241ae844d28-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.259919 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.259784    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2k7d\" (UniqueName: \"kubernetes.io/projected/3d473c20-cdc7-45e7-9fb8-b241ae844d28-kube-api-access-k2k7d\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sbf5z\" (UID: \"3d473c20-cdc7-45e7-9fb8-b241ae844d28\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.352187 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.352095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:10.533518 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:30:10.533478    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d473c20_cdc7_45e7_9fb8_b241ae844d28.slice/crio-ca12e9c732f22a39cdb1103407ecaa986bbadcf5c19c361eeed5435ed74d04f9 WatchSource:0}: Error finding container ca12e9c732f22a39cdb1103407ecaa986bbadcf5c19c361eeed5435ed74d04f9: Status 404 returned error can't find the container with id ca12e9c732f22a39cdb1103407ecaa986bbadcf5c19c361eeed5435ed74d04f9
Jul 09 12:30:10.538344 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:10.538320    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"]
Jul 09 12:30:11.496712 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:11.496660    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:11.513083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:11.513040    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z" event={"ID":"3d473c20-cdc7-45e7-9fb8-b241ae844d28","Type":"ContainerStarted","Data":"ca12e9c732f22a39cdb1103407ecaa986bbadcf5c19c361eeed5435ed74d04f9"}
Jul 09 12:30:12.172373 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.172335    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"]
Jul 09 12:30:12.172594 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.172544    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerName="discovery" containerID="cri-o://3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1" gracePeriod=30
Jul 09 12:30:12.806905 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.806871    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"]
Jul 09 12:30:12.815489 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.815449    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"]
Jul 09 12:30:12.815668 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.815583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:12.819234 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.819206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.819420 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.819400    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jvs7f\""
Jul 09 12:30:12.819420 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.819413    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 09 12:30:12.828377 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.828350    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 09 12:30:12.828516 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.828477    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2qd6f\""
Jul 09 12:30:12.845070 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.845027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7wdl\" (UniqueName: \"kubernetes.io/projected/cbccf436-b3e3-4ea7-b8be-0108bfe60a82-kube-api-access-r7wdl\") pod \"llmisvc-controller-manager-777c6bd7d5-fzbfg\" (UID: \"cbccf436-b3e3-4ea7-b8be-0108bfe60a82\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.845202 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.845082    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a979de88-4fdf-45a7-86c3-8fc900949cd6-cert\") pod \"kserve-controller-manager-d6f4b65b8-hrwmn\" (UID: \"a979de88-4fdf-45a7-86c3-8fc900949cd6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:12.845202 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.845132    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbccf436-b3e3-4ea7-b8be-0108bfe60a82-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fzbfg\" (UID: \"cbccf436-b3e3-4ea7-b8be-0108bfe60a82\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.845321 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.845217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd9hv\" (UniqueName: \"kubernetes.io/projected/a979de88-4fdf-45a7-86c3-8fc900949cd6-kube-api-access-rd9hv\") pod \"kserve-controller-manager-d6f4b65b8-hrwmn\" (UID: \"a979de88-4fdf-45a7-86c3-8fc900949cd6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:12.846520 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.846496    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"]
Jul 09 12:30:12.858481 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.858448    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"]
Jul 09 12:30:12.890450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.890409    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:30:12.890595 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.890480    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:30:12.890595 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.890506    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:30:12.945910 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.945882    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbccf436-b3e3-4ea7-b8be-0108bfe60a82-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fzbfg\" (UID: \"cbccf436-b3e3-4ea7-b8be-0108bfe60a82\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.946026 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.945943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd9hv\" (UniqueName: \"kubernetes.io/projected/a979de88-4fdf-45a7-86c3-8fc900949cd6-kube-api-access-rd9hv\") pod \"kserve-controller-manager-d6f4b65b8-hrwmn\" (UID: \"a979de88-4fdf-45a7-86c3-8fc900949cd6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:12.946026 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.946012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7wdl\" (UniqueName: \"kubernetes.io/projected/cbccf436-b3e3-4ea7-b8be-0108bfe60a82-kube-api-access-r7wdl\") pod \"llmisvc-controller-manager-777c6bd7d5-fzbfg\" (UID: \"cbccf436-b3e3-4ea7-b8be-0108bfe60a82\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.946096 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.946053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a979de88-4fdf-45a7-86c3-8fc900949cd6-cert\") pod \"kserve-controller-manager-d6f4b65b8-hrwmn\" (UID: \"a979de88-4fdf-45a7-86c3-8fc900949cd6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:12.948523 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.948498    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbccf436-b3e3-4ea7-b8be-0108bfe60a82-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fzbfg\" (UID: \"cbccf436-b3e3-4ea7-b8be-0108bfe60a82\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.948765 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.948744    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a979de88-4fdf-45a7-86c3-8fc900949cd6-cert\") pod \"kserve-controller-manager-d6f4b65b8-hrwmn\" (UID: \"a979de88-4fdf-45a7-86c3-8fc900949cd6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:12.976298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.976267    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:12.980697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.980652    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7wdl\" (UniqueName: \"kubernetes.io/projected/cbccf436-b3e3-4ea7-b8be-0108bfe60a82-kube-api-access-r7wdl\") pod \"llmisvc-controller-manager-777c6bd7d5-fzbfg\" (UID: \"cbccf436-b3e3-4ea7-b8be-0108bfe60a82\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:12.981608 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:12.981584    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd9hv\" (UniqueName: \"kubernetes.io/projected/a979de88-4fdf-45a7-86c3-8fc900949cd6-kube-api-access-rd9hv\") pod \"kserve-controller-manager-d6f4b65b8-hrwmn\" (UID: \"a979de88-4fdf-45a7-86c3-8fc900949cd6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:13.047216 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047180    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdkcf\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-kube-api-access-pdkcf\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.047401 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047263    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-kubeconfig\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.047401 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047296    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-token\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.047401 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047341    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-dns-cert\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.047401 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047364    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-cacerts\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.047619 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047423    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-local-certs\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.047619 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.047512    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-ca-configmap\") pod \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\" (UID: \"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3\") "
Jul 09 12:30:13.048221 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.048174    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:30:13.048621 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.048605    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-ca-configmap\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.050258 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.050231    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:30:13.050362 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.050318    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-cacerts" (OuterVolumeSpecName: "cacerts") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:30:13.050415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.050395    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-kube-api-access-pdkcf" (OuterVolumeSpecName: "kube-api-access-pdkcf") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "kube-api-access-pdkcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:30:13.050469 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.050451    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:30:13.050649 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.050622    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:30:13.051097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.051080    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-local-certs" (OuterVolumeSpecName: "local-certs") pod "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" (UID: "76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:30:13.137768 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.137654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:13.143942 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.143912    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:13.150152 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.150119    2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-kubeconfig\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.150269 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.150155    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.150269 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.150170    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-istio-csr-dns-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.150269 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.150184    2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-cacerts\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.150269 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.150200    2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-local-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.150269 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.150215    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdkcf\" (UniqueName: \"kubernetes.io/projected/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3-kube-api-access-pdkcf\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:30:13.404100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.404069    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"]
Jul 09 12:30:13.405845 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:30:13.405817    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda979de88_4fdf_45a7_86c3_8fc900949cd6.slice/crio-204bc2f0899674f60b14beda112063e24e8ece169c0c5c97062aa2f9ff266cf1 WatchSource:0}: Error finding container 204bc2f0899674f60b14beda112063e24e8ece169c0c5c97062aa2f9ff266cf1: Status 404 returned error can't find the container with id 204bc2f0899674f60b14beda112063e24e8ece169c0c5c97062aa2f9ff266cf1
Jul 09 12:30:13.409079 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.409057    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"]
Jul 09 12:30:13.412200 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:30:13.412162    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcbccf436_b3e3_4ea7_b8be_0108bfe60a82.slice/crio-6e87d9bc6ef3f898950352d280c01a619b224b9215e095eed304e9ced2056df8 WatchSource:0}: Error finding container 6e87d9bc6ef3f898950352d280c01a619b224b9215e095eed304e9ced2056df8: Status 404 returned error can't find the container with id 6e87d9bc6ef3f898950352d280c01a619b224b9215e095eed304e9ced2056df8
Jul 09 12:30:13.472853 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.472816    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gpsh7"
Jul 09 12:30:13.474582 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.474559    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8m6g5"
Jul 09 12:30:13.524219 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.524186    2574 generic.go:356] "Generic (PLEG): container finished" podID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerID="3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1" exitCode=0
Jul 09 12:30:13.524402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.524231    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"
Jul 09 12:30:13.524402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.524278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" event={"ID":"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3","Type":"ContainerDied","Data":"3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1"}
Jul 09 12:30:13.524402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.524328    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8" event={"ID":"76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3","Type":"ContainerDied","Data":"46cccc66882257dcd6ce47d01a35a6dee3bda67d3e6fe7eba1dd9b5854a263c3"}
Jul 09 12:30:13.524402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.524355    2574 scope.go:117] "RemoveContainer" containerID="3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1"
Jul 09 12:30:13.525725 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.525698    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg" event={"ID":"cbccf436-b3e3-4ea7-b8be-0108bfe60a82","Type":"ContainerStarted","Data":"6e87d9bc6ef3f898950352d280c01a619b224b9215e095eed304e9ced2056df8"}
Jul 09 12:30:13.527312 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.527278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z" event={"ID":"3d473c20-cdc7-45e7-9fb8-b241ae844d28","Type":"ContainerStarted","Data":"eee9c695165aa3502bb0a74c18e50d45f21826241616778a8483cc9044091eb4"}
Jul 09 12:30:13.528580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.528557    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn" event={"ID":"a979de88-4fdf-45a7-86c3-8fc900949cd6","Type":"ContainerStarted","Data":"204bc2f0899674f60b14beda112063e24e8ece169c0c5c97062aa2f9ff266cf1"}
Jul 09 12:30:13.533175 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.533157    2574 scope.go:117] "RemoveContainer" containerID="3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1"
Jul 09 12:30:13.533443 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:30:13.533425    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1\": container with ID starting with 3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1 not found: ID does not exist" containerID="3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1"
Jul 09 12:30:13.533486 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.533452    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1"} err="failed to get container status \"3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1\": rpc error: code = NotFound desc = could not find container \"3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1\": container with ID starting with 3767cde5a71564c6b2a840f9e16ba7d70951ab5fa54e614b39e62f5ce2e642a1 not found: ID does not exist"
Jul 09 12:30:13.688278 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.688199    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"]
Jul 09 12:30:13.696005 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:13.695978    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4xpj8"]
Jul 09 12:30:14.352935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:14.352902    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:14.358365 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:14.358339    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:14.427370 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:14.427295    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z" podStartSLOduration=3.072678728 podStartE2EDuration="5.427271469s" totalImagesPullingTime="2.354592741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:13.739527202 +0000 UTC m=+371.334151273" watchObservedRunningTime="2026-07-09 12:30:14.427271469 +0000 UTC m=+372.021895513"
Jul 09 12:30:14.538942 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:14.538912    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:14.540170 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:14.540127    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sbf5z"
Jul 09 12:30:14.970886 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:14.970851    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" path="/var/lib/kubelet/pods/76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3/volumes"
Jul 09 12:30:17.555852 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:17.555812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg" event={"ID":"cbccf436-b3e3-4ea7-b8be-0108bfe60a82","Type":"ContainerStarted","Data":"04802ca0be41e6c92889b665e28380e2a1a2d6efe90f8e2b7b72d2c83bd4219c"}
Jul 09 12:30:17.556326 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:17.555882    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:17.557255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:17.557230    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn" event={"ID":"a979de88-4fdf-45a7-86c3-8fc900949cd6","Type":"ContainerStarted","Data":"5a0acc237378dce2eb208f57903db283a280df2ea025b55ce92a4c916e480c7f"}
Jul 09 12:30:17.557390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:17.557376    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:17.598178 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:17.598125    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg" podStartSLOduration=2.253976287 podStartE2EDuration="5.598105947s" totalImagesPullingTime="3.34412966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:17.597113989 +0000 UTC m=+375.191738033" watchObservedRunningTime="2026-07-09 12:30:17.598105947 +0000 UTC m=+375.192729993"
Jul 09 12:30:17.637250 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:17.637193    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn" podStartSLOduration=2.337856201 podStartE2EDuration="5.637177662s" totalImagesPullingTime="3.299321461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:17.636262335 +0000 UTC m=+375.230886379" watchObservedRunningTime="2026-07-09 12:30:17.637177662 +0000 UTC m=+375.231801704"
Jul 09 12:30:48.563832 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:48.563792    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fzbfg"
Jul 09 12:30:48.566783 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:48.566760    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hrwmn"
Jul 09 12:30:55.926633 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.926596    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b7l49"]
Jul 09 12:30:55.927127 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.927059    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerName="discovery"
Jul 09 12:30:55.927127 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.927077    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerName="discovery"
Jul 09 12:30:55.927250 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.927183    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d3d7a2-07cc-4a5f-a996-aa6b2c2e72a3" containerName="discovery"
Jul 09 12:30:55.938119 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.938093    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:30:55.940771 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.940711    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 12:30:55.940771 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.940721    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 12:30:55.941650 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.941630    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zd4rg\""
Jul 09 12:30:55.947848 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:55.947820    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b7l49"]
Jul 09 12:30:56.015158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:56.015122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9wtq\" (UniqueName: \"kubernetes.io/projected/cda6b1a9-2f83-41ae-92eb-1d7538c47b3e-kube-api-access-m9wtq\") pod \"kuadrant-operator-catalog-b7l49\" (UID: \"cda6b1a9-2f83-41ae-92eb-1d7538c47b3e\") " pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:30:56.116327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:56.116294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9wtq\" (UniqueName: \"kubernetes.io/projected/cda6b1a9-2f83-41ae-92eb-1d7538c47b3e-kube-api-access-m9wtq\") pod \"kuadrant-operator-catalog-b7l49\" (UID: \"cda6b1a9-2f83-41ae-92eb-1d7538c47b3e\") " pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:30:56.129239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:56.129208    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9wtq\" (UniqueName: \"kubernetes.io/projected/cda6b1a9-2f83-41ae-92eb-1d7538c47b3e-kube-api-access-m9wtq\") pod \"kuadrant-operator-catalog-b7l49\" (UID: \"cda6b1a9-2f83-41ae-92eb-1d7538c47b3e\") " pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:30:56.248450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:56.248420    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:30:56.381279 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:56.381255    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b7l49"]
Jul 09 12:30:56.383976 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:30:56.383945    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda6b1a9_2f83_41ae_92eb_1d7538c47b3e.slice/crio-aae886248a44d454afd1fab15a047e207059186bd8e3765fe8d06a4d27c6cd85 WatchSource:0}: Error finding container aae886248a44d454afd1fab15a047e207059186bd8e3765fe8d06a4d27c6cd85: Status 404 returned error can't find the container with id aae886248a44d454afd1fab15a047e207059186bd8e3765fe8d06a4d27c6cd85
Jul 09 12:30:56.727980 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:56.727948    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b7l49" event={"ID":"cda6b1a9-2f83-41ae-92eb-1d7538c47b3e","Type":"ContainerStarted","Data":"aae886248a44d454afd1fab15a047e207059186bd8e3765fe8d06a4d27c6cd85"}
Jul 09 12:30:58.739459 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:58.739420    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b7l49" event={"ID":"cda6b1a9-2f83-41ae-92eb-1d7538c47b3e","Type":"ContainerStarted","Data":"7c053dd5f25bc00ded149b20aa40abee4b7b9c74c261fcb2f152fa59355c2c33"}
Jul 09 12:30:58.769871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:30:58.769821    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b7l49" podStartSLOduration=1.599115498 podStartE2EDuration="3.76980617s" totalImagesPullingTime="2.170690672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:30:58.76801216 +0000 UTC m=+416.362636215" watchObservedRunningTime="2026-07-09 12:30:58.76980617 +0000 UTC m=+416.364430217"
Jul 09 12:31:06.248983 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:06.248946    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:31:06.249382 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:06.249176    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:31:06.271732 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:06.271708    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:31:06.796255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:06.796227    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-b7l49"
Jul 09 12:31:23.613201 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.613161    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59cd66ddfc-hmcnv"]
Jul 09 12:31:23.618393 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.618371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.622437 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.622415    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 12:31:23.623407 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.623384    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 12:31:23.623523 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.623394    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 12:31:23.623591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.623394    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 12:31:23.624551 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.624531    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nj5qb\""
Jul 09 12:31:23.624735 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.624636    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 12:31:23.627308 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.627284    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 12:31:23.630155 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.630114    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cd66ddfc-hmcnv"]
Jul 09 12:31:23.767736 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e5a5bd-f739-4634-abbc-8f57ab488531-console-serving-cert\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.767916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tm85\" (UniqueName: \"kubernetes.io/projected/19e5a5bd-f739-4634-abbc-8f57ab488531-kube-api-access-4tm85\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.767916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-console-config\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.767916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-service-ca\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.768016 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e5a5bd-f739-4634-abbc-8f57ab488531-console-oauth-config\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.768016 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-oauth-serving-cert\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.768016 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.767978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-trusted-ca-bundle\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869256 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e5a5bd-f739-4634-abbc-8f57ab488531-console-oauth-config\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869256 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869190    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-oauth-serving-cert\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869256 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-trusted-ca-bundle\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869256 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869233    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e5a5bd-f739-4634-abbc-8f57ab488531-console-serving-cert\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tm85\" (UniqueName: \"kubernetes.io/projected/19e5a5bd-f739-4634-abbc-8f57ab488531-kube-api-access-4tm85\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-console-config\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.869563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.869493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-service-ca\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.870208 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.870178    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-oauth-serving-cert\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.870284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.870217    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-service-ca\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.870284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.870233    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-console-config\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.870829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.870809    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e5a5bd-f739-4634-abbc-8f57ab488531-trusted-ca-bundle\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.871929 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.871908    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e5a5bd-f739-4634-abbc-8f57ab488531-console-serving-cert\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.872060 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.872039    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e5a5bd-f739-4634-abbc-8f57ab488531-console-oauth-config\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.880966 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.880947    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tm85\" (UniqueName: \"kubernetes.io/projected/19e5a5bd-f739-4634-abbc-8f57ab488531-kube-api-access-4tm85\") pod \"console-59cd66ddfc-hmcnv\" (UID: \"19e5a5bd-f739-4634-abbc-8f57ab488531\") " pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:23.928476 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:23.928443    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:24.094885 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:24.094852    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cd66ddfc-hmcnv"]
Jul 09 12:31:24.098377 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:24.098354    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e5a5bd_f739_4634_abbc_8f57ab488531.slice/crio-46290b3ef80e6ca30458a8762a42c4aa3bad932500e58cd1f428841070025d8c WatchSource:0}: Error finding container 46290b3ef80e6ca30458a8762a42c4aa3bad932500e58cd1f428841070025d8c: Status 404 returned error can't find the container with id 46290b3ef80e6ca30458a8762a42c4aa3bad932500e58cd1f428841070025d8c
Jul 09 12:31:24.860353 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:24.860316    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cd66ddfc-hmcnv" event={"ID":"19e5a5bd-f739-4634-abbc-8f57ab488531","Type":"ContainerStarted","Data":"d65b72897019b5249d16fb57ec6caab60e7752b8659f76d96989b56b18e39371"}
Jul 09 12:31:24.860763 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:24.860365    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cd66ddfc-hmcnv" event={"ID":"19e5a5bd-f739-4634-abbc-8f57ab488531","Type":"ContainerStarted","Data":"46290b3ef80e6ca30458a8762a42c4aa3bad932500e58cd1f428841070025d8c"}
Jul 09 12:31:24.883741 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:24.883662    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59cd66ddfc-hmcnv" podStartSLOduration=1.883645851 podStartE2EDuration="1.883645851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:24.883355668 +0000 UTC m=+442.477979714" watchObservedRunningTime="2026-07-09 12:31:24.883645851 +0000 UTC m=+442.478269872"
Jul 09 12:31:28.919770 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:28.919729    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"]
Jul 09 12:31:28.926449 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:28.926418    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:28.930003 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:28.929980    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m247t\""
Jul 09 12:31:28.942017 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:28.941989    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"]
Jul 09 12:31:29.117760 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.117717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b314636-f7e1-4f70-8cd8-402c4abe91cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.117939 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.117796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx4m9\" (UniqueName: \"kubernetes.io/projected/8b314636-f7e1-4f70-8cd8-402c4abe91cf-kube-api-access-jx4m9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.218509 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.218471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b314636-f7e1-4f70-8cd8-402c4abe91cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.218641 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.218527    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx4m9\" (UniqueName: \"kubernetes.io/projected/8b314636-f7e1-4f70-8cd8-402c4abe91cf-kube-api-access-jx4m9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.218867 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.218847    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b314636-f7e1-4f70-8cd8-402c4abe91cf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.231255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.231217    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx4m9\" (UniqueName: \"kubernetes.io/projected/8b314636-f7e1-4f70-8cd8-402c4abe91cf-kube-api-access-jx4m9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.236813 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.236792    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:29.382636 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.382601    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"]
Jul 09 12:31:29.385619 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:29.385593    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b314636_f7e1_4f70_8cd8_402c4abe91cf.slice/crio-8145efdabe1cf44fce77b7864a17f303e686bc730c878f8e9352c632f22c37d2 WatchSource:0}: Error finding container 8145efdabe1cf44fce77b7864a17f303e686bc730c878f8e9352c632f22c37d2: Status 404 returned error can't find the container with id 8145efdabe1cf44fce77b7864a17f303e686bc730c878f8e9352c632f22c37d2
Jul 09 12:31:29.885758 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:29.885720    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk" event={"ID":"8b314636-f7e1-4f70-8cd8-402c4abe91cf","Type":"ContainerStarted","Data":"8145efdabe1cf44fce77b7864a17f303e686bc730c878f8e9352c632f22c37d2"}
Jul 09 12:31:31.438441 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.438402    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"]
Jul 09 12:31:31.442094 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.442063    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:31.445549 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.445525    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rzv5f\""
Jul 09 12:31:31.457674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.457646    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"]
Jul 09 12:31:31.541633 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.541590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p56xs\" (UniqueName: \"kubernetes.io/projected/6691dac7-b236-45c3-a63b-8adbbd08266b-kube-api-access-p56xs\") pod \"authorino-operator-75cc8d5cb7-stw7q\" (UID: \"6691dac7-b236-45c3-a63b-8adbbd08266b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:31.642636 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.642583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p56xs\" (UniqueName: \"kubernetes.io/projected/6691dac7-b236-45c3-a63b-8adbbd08266b-kube-api-access-p56xs\") pod \"authorino-operator-75cc8d5cb7-stw7q\" (UID: \"6691dac7-b236-45c3-a63b-8adbbd08266b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:31.652440 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.652401    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p56xs\" (UniqueName: \"kubernetes.io/projected/6691dac7-b236-45c3-a63b-8adbbd08266b-kube-api-access-p56xs\") pod \"authorino-operator-75cc8d5cb7-stw7q\" (UID: \"6691dac7-b236-45c3-a63b-8adbbd08266b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:31.755583 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.755546    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:31.932253 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:31.932228    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"]
Jul 09 12:31:31.934111 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:31.934081    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6691dac7_b236_45c3_a63b_8adbbd08266b.slice/crio-8caa7b1119b1ad911d7b9b6bdc2ec387e76f803fdfdedc792208b5e08a2f5579 WatchSource:0}: Error finding container 8caa7b1119b1ad911d7b9b6bdc2ec387e76f803fdfdedc792208b5e08a2f5579: Status 404 returned error can't find the container with id 8caa7b1119b1ad911d7b9b6bdc2ec387e76f803fdfdedc792208b5e08a2f5579
Jul 09 12:31:32.905646 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:32.905570    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q" event={"ID":"6691dac7-b236-45c3-a63b-8adbbd08266b","Type":"ContainerStarted","Data":"8caa7b1119b1ad911d7b9b6bdc2ec387e76f803fdfdedc792208b5e08a2f5579"}
Jul 09 12:31:33.709155 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.709095    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"]
Jul 09 12:31:33.715189 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.715162    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:33.718403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.718377    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 12:31:33.718552 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.718497    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rfb6j\""
Jul 09 12:31:33.733555 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.733525    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"]
Jul 09 12:31:33.760039 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.760003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6hmc\" (UniqueName: \"kubernetes.io/projected/f07563b9-bc90-4fe9-a2cd-3dec3a121347-kube-api-access-f6hmc\") pod \"dns-operator-controller-manager-864dd78c4c-5c6wp\" (UID: \"f07563b9-bc90-4fe9-a2cd-3dec3a121347\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:33.860956 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.860917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6hmc\" (UniqueName: \"kubernetes.io/projected/f07563b9-bc90-4fe9-a2cd-3dec3a121347-kube-api-access-f6hmc\") pod \"dns-operator-controller-manager-864dd78c4c-5c6wp\" (UID: \"f07563b9-bc90-4fe9-a2cd-3dec3a121347\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:33.872255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.872225    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6hmc\" (UniqueName: \"kubernetes.io/projected/f07563b9-bc90-4fe9-a2cd-3dec3a121347-kube-api-access-f6hmc\") pod \"dns-operator-controller-manager-864dd78c4c-5c6wp\" (UID: \"f07563b9-bc90-4fe9-a2cd-3dec3a121347\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:33.929690 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.929507    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:33.929690 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.929626    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:33.936218 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:33.936115    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:34.029080 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.028987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:34.287975 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.287952    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"]
Jul 09 12:31:34.550259 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:34.550185    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07563b9_bc90_4fe9_a2cd_3dec3a121347.slice/crio-3807f6f914e77b18e763716825cf3c8d7e7cd85ad9bd95ca7cdd09656bea5e6a WatchSource:0}: Error finding container 3807f6f914e77b18e763716825cf3c8d7e7cd85ad9bd95ca7cdd09656bea5e6a: Status 404 returned error can't find the container with id 3807f6f914e77b18e763716825cf3c8d7e7cd85ad9bd95ca7cdd09656bea5e6a
Jul 09 12:31:34.917730 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.917613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q" event={"ID":"6691dac7-b236-45c3-a63b-8adbbd08266b","Type":"ContainerStarted","Data":"f182708aef72366cbe37ee7215d7ed758f0b9424dd30453902633e2625bb7218"}
Jul 09 12:31:34.917910 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.917818    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:34.919010 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.918988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp" event={"ID":"f07563b9-bc90-4fe9-a2cd-3dec3a121347","Type":"ContainerStarted","Data":"3807f6f914e77b18e763716825cf3c8d7e7cd85ad9bd95ca7cdd09656bea5e6a"}
Jul 09 12:31:34.923017 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.922994    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59cd66ddfc-hmcnv"
Jul 09 12:31:34.938264 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:34.938203    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q" podStartSLOduration=1.288441384 podStartE2EDuration="3.938188649s" totalImagesPullingTime="2.649747265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:34.936988723 +0000 UTC m=+452.531612785" watchObservedRunningTime="2026-07-09 12:31:34.938188649 +0000 UTC m=+452.532812691"
Jul 09 12:31:35.737409 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.737367    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"]
Jul 09 12:31:35.740251 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.740225    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:35.743052 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.743023    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8x7jw\""
Jul 09 12:31:35.752771 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.752741    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"]
Jul 09 12:31:35.775814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.775777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsfbn\" (UniqueName: \"kubernetes.io/projected/2fc11349-796a-46f7-b2fa-76a9f1775933-kube-api-access-gsfbn\") pod \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" (UID: \"2fc11349-796a-46f7-b2fa-76a9f1775933\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:35.876546 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.876503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfbn\" (UniqueName: \"kubernetes.io/projected/2fc11349-796a-46f7-b2fa-76a9f1775933-kube-api-access-gsfbn\") pod \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" (UID: \"2fc11349-796a-46f7-b2fa-76a9f1775933\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:35.886444 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:35.886418    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsfbn\" (UniqueName: \"kubernetes.io/projected/2fc11349-796a-46f7-b2fa-76a9f1775933-kube-api-access-gsfbn\") pod \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" (UID: \"2fc11349-796a-46f7-b2fa-76a9f1775933\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:36.053939 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:36.053852    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:36.931616 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:36.931593    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"]
Jul 09 12:31:36.944909 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:36.944791    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc11349_796a_46f7_b2fa_76a9f1775933.slice/crio-19ef871eba61d84eebbda7db148263b3720746a0ae28bafbca5cdabc8ea58cb5 WatchSource:0}: Error finding container 19ef871eba61d84eebbda7db148263b3720746a0ae28bafbca5cdabc8ea58cb5: Status 404 returned error can't find the container with id 19ef871eba61d84eebbda7db148263b3720746a0ae28bafbca5cdabc8ea58cb5
Jul 09 12:31:37.942333 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.942252    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb" event={"ID":"2fc11349-796a-46f7-b2fa-76a9f1775933","Type":"ContainerStarted","Data":"19ef871eba61d84eebbda7db148263b3720746a0ae28bafbca5cdabc8ea58cb5"}
Jul 09 12:31:37.944804 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.944728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp" event={"ID":"f07563b9-bc90-4fe9-a2cd-3dec3a121347","Type":"ContainerStarted","Data":"04de0eb6699c1fae39467ba67d64ea6b6bdc26f844a20fce10a8d0cc7dc6e792"}
Jul 09 12:31:37.945025 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.944971    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:37.946988 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.946942    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk" event={"ID":"8b314636-f7e1-4f70-8cd8-402c4abe91cf","Type":"ContainerStarted","Data":"fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160"}
Jul 09 12:31:37.947096 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.947000    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:37.965939 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.965886    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp" podStartSLOduration=2.131497942 podStartE2EDuration="4.965868831s" totalImagesPullingTime="2.834370889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:37.964766693 +0000 UTC m=+455.559390761" watchObservedRunningTime="2026-07-09 12:31:37.965868831 +0000 UTC m=+455.560492873"
Jul 09 12:31:37.992309 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:37.992254    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk" podStartSLOduration=2.439852676 podStartE2EDuration="9.992235702s" totalImagesPullingTime="7.552383026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:37.990079063 +0000 UTC m=+455.584703125" watchObservedRunningTime="2026-07-09 12:31:37.992235702 +0000 UTC m=+455.586859744"
Jul 09 12:31:38.954877 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:38.954837    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb" event={"ID":"2fc11349-796a-46f7-b2fa-76a9f1775933","Type":"ContainerStarted","Data":"52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25"}
Jul 09 12:31:38.955338 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:38.955078    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:38.977392 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:38.977337    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb" podStartSLOduration=2.067214902 podStartE2EDuration="3.977320344s" totalImagesPullingTime="1.910105442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:38.976283816 +0000 UTC m=+456.570907859" watchObservedRunningTime="2026-07-09 12:31:38.977320344 +0000 UTC m=+456.571944386"
Jul 09 12:31:45.928052 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:45.928019    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-stw7q"
Jul 09 12:31:48.957670 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:48.957637    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5c6wp"
Jul 09 12:31:48.958154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:48.957732    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:49.962955 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:49.962924    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:50.992453 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:50.992418    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"]
Jul 09 12:31:50.992875 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:50.992656    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" containerName="manager" containerID="cri-o://fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160" gracePeriod=2
Jul 09 12:31:51.004791 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.004763    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"]
Jul 09 12:31:51.016403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.016374    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"]
Jul 09 12:31:51.016607 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.016580    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" containerName="manager" containerID="cri-o://52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25" gracePeriod=2
Jul 09 12:31:51.021702 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.020977    2574 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8b314636-f7e1-4f70-8cd8-402c4abe91cf\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\": pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.022934 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.022900    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.023213 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.023194    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"]
Jul 09 12:31:51.023899 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.023878    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" containerName="manager"
Jul 09 12:31:51.023899 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.023901    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" containerName="manager"
Jul 09 12:31:51.024045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.023987    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" containerName="manager"
Jul 09 12:31:51.025289 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.025262    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:51.026591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.026562    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"]
Jul 09 12:31:51.026744 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.026726    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.028797 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.028773    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.030598 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.030566    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:51.043715 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.043669    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"]
Jul 09 12:31:51.046166 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.046137    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"]
Jul 09 12:31:51.046657 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.046640    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" containerName="manager"
Jul 09 12:31:51.046726 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.046662    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" containerName="manager"
Jul 09 12:31:51.046803 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.046791    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" containerName="manager"
Jul 09 12:31:51.048551 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.048533    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:31:51.093760 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.093732    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"]
Jul 09 12:31:51.098119 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.098088    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.099914 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.099884    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:51.112981 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.112952    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/378c0159-769a-4711-878f-a8c5978eea2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k7rgx\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.113073 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.112985    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5rt9\" (UniqueName: \"kubernetes.io/projected/5cc855e4-d845-4bd1-9098-b653405552cf-kube-api-access-r5rt9\") pod \"limitador-operator-controller-manager-859d7b78bd-dltzc\" (UID: \"5cc855e4-d845-4bd1-9098-b653405552cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:31:51.113073 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.113022    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5v4l\" (UniqueName: \"kubernetes.io/projected/378c0159-769a-4711-878f-a8c5978eea2c-kube-api-access-j5v4l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k7rgx\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.214570 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.214535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5v4l\" (UniqueName: \"kubernetes.io/projected/378c0159-769a-4711-878f-a8c5978eea2c-kube-api-access-j5v4l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k7rgx\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.214759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.214659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/378c0159-769a-4711-878f-a8c5978eea2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k7rgx\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.214759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.214706    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5rt9\" (UniqueName: \"kubernetes.io/projected/5cc855e4-d845-4bd1-9098-b653405552cf-kube-api-access-r5rt9\") pod \"limitador-operator-controller-manager-859d7b78bd-dltzc\" (UID: \"5cc855e4-d845-4bd1-9098-b653405552cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:31:51.215072 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.215049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/378c0159-769a-4711-878f-a8c5978eea2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k7rgx\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.230261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.230228    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5rt9\" (UniqueName: \"kubernetes.io/projected/5cc855e4-d845-4bd1-9098-b653405552cf-kube-api-access-r5rt9\") pod \"limitador-operator-controller-manager-859d7b78bd-dltzc\" (UID: \"5cc855e4-d845-4bd1-9098-b653405552cf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:31:51.230434 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.230409    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5v4l\" (UniqueName: \"kubernetes.io/projected/378c0159-769a-4711-878f-a8c5978eea2c-kube-api-access-j5v4l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k7rgx\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.262081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.262059    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:51.264836 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.264620    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.266879 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.266852    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:51.267895 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.267877    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.269805 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.269780    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:51.271609 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:51.271592    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:51.315881 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.315848    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b314636-f7e1-4f70-8cd8-402c4abe91cf-extensions-socket-volume\") pod \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") "
Jul 09 12:31:51.316089 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.315959    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx4m9\" (UniqueName: \"kubernetes.io/projected/8b314636-f7e1-4f70-8cd8-402c4abe91cf-kube-api-access-jx4m9\") pod \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\" (UID: \"8b314636-f7e1-4f70-8cd8-402c4abe91cf\") "
Jul 09 12:31:51.316089 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.315999    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsfbn\" (UniqueName: \"kubernetes.io/projected/2fc11349-796a-46f7-b2fa-76a9f1775933-kube-api-access-gsfbn\") pod \"2fc11349-796a-46f7-b2fa-76a9f1775933\" (UID: \"2fc11349-796a-46f7-b2fa-76a9f1775933\") "
Jul 09 12:31:51.316450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.316419    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b314636-f7e1-4f70-8cd8-402c4abe91cf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8b314636-f7e1-4f70-8cd8-402c4abe91cf" (UID: "8b314636-f7e1-4f70-8cd8-402c4abe91cf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:31:51.318347 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.318275    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc11349-796a-46f7-b2fa-76a9f1775933-kube-api-access-gsfbn" (OuterVolumeSpecName: "kube-api-access-gsfbn") pod "2fc11349-796a-46f7-b2fa-76a9f1775933" (UID: "2fc11349-796a-46f7-b2fa-76a9f1775933"). InnerVolumeSpecName "kube-api-access-gsfbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:31:51.318506 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.318489    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b314636-f7e1-4f70-8cd8-402c4abe91cf-kube-api-access-jx4m9" (OuterVolumeSpecName: "kube-api-access-jx4m9") pod "8b314636-f7e1-4f70-8cd8-402c4abe91cf" (UID: "8b314636-f7e1-4f70-8cd8-402c4abe91cf"). InnerVolumeSpecName "kube-api-access-jx4m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:31:51.416944 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.416897    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx4m9\" (UniqueName: \"kubernetes.io/projected/8b314636-f7e1-4f70-8cd8-402c4abe91cf-kube-api-access-jx4m9\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:31:51.416944 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.416941    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsfbn\" (UniqueName: \"kubernetes.io/projected/2fc11349-796a-46f7-b2fa-76a9f1775933-kube-api-access-gsfbn\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:31:51.416944 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.416951    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b314636-f7e1-4f70-8cd8-402c4abe91cf-extensions-socket-volume\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:31:51.446854 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.446814    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:51.453728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.453704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:31:51.634389 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.634356    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"]
Jul 09 12:31:51.634518 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:51.634444    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc855e4_d845_4bd1_9098_b653405552cf.slice/crio-d28273ad5345af3b4f99a31da9b2e4a338ab8be762994652b43d61d2bb108ebb WatchSource:0}: Error finding container d28273ad5345af3b4f99a31da9b2e4a338ab8be762994652b43d61d2bb108ebb: Status 404 returned error can't find the container with id d28273ad5345af3b4f99a31da9b2e4a338ab8be762994652b43d61d2bb108ebb
Jul 09 12:31:51.650270 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:51.650244    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"]
Jul 09 12:31:51.656800 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:31:51.656766    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378c0159_769a_4711_878f_a8c5978eea2c.slice/crio-a20b8d56426a8cce132cc730b9e26119b5555fb33cabd3c17efaf62a91e4d575 WatchSource:0}: Error finding container a20b8d56426a8cce132cc730b9e26119b5555fb33cabd3c17efaf62a91e4d575: Status 404 returned error can't find the container with id a20b8d56426a8cce132cc730b9e26119b5555fb33cabd3c17efaf62a91e4d575
Jul 09 12:31:52.022498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.022462    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc" event={"ID":"5cc855e4-d845-4bd1-9098-b653405552cf","Type":"ContainerStarted","Data":"336477529c6bec696a6910519ad4052246431413b739d050bd5d9701740167ce"}
Jul 09 12:31:52.022940 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.022522    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc" event={"ID":"5cc855e4-d845-4bd1-9098-b653405552cf","Type":"ContainerStarted","Data":"d28273ad5345af3b4f99a31da9b2e4a338ab8be762994652b43d61d2bb108ebb"}
Jul 09 12:31:52.022940 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.022545    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:31:52.024182 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.024160    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx" event={"ID":"378c0159-769a-4711-878f-a8c5978eea2c","Type":"ContainerStarted","Data":"2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d"}
Jul 09 12:31:52.024305 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.024198    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx" event={"ID":"378c0159-769a-4711-878f-a8c5978eea2c","Type":"ContainerStarted","Data":"a20b8d56426a8cce132cc730b9e26119b5555fb33cabd3c17efaf62a91e4d575"}
Jul 09 12:31:52.024305 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.024256    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:31:52.025413 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.025394    2574 generic.go:356] "Generic (PLEG): container finished" podID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" containerID="fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160" exitCode=0
Jul 09 12:31:52.025476 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.025433    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:52.025563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.025433    2574 scope.go:117] "RemoveContainer" containerID="fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160"
Jul 09 12:31:52.026787 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.026769    2574 generic.go:356] "Generic (PLEG): container finished" podID="2fc11349-796a-46f7-b2fa-76a9f1775933" containerID="52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25" exitCode=0
Jul 09 12:31:52.026837 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.026817    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:52.034931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.034229    2574 scope.go:117] "RemoveContainer" containerID="fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160"
Jul 09 12:31:52.034931 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.034653    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160\": container with ID starting with fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160 not found: ID does not exist" containerID="fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160"
Jul 09 12:31:52.034931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.034712    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160"} err="failed to get container status \"fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160\": rpc error: code = NotFound desc = could not find container \"fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160\": container with ID starting with fb21a70656e5a141ef704f74099b0fa1f8028faae551efde85dc6cd84ca4f160 not found: ID does not exist"
Jul 09 12:31:52.034931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.034739    2574 scope.go:117] "RemoveContainer" containerID="52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25"
Jul 09 12:31:52.044471 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.044423    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc" podStartSLOduration=1.044408734 podStartE2EDuration="1.044408734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:52.043849273 +0000 UTC m=+469.638473318" watchObservedRunningTime="2026-07-09 12:31:52.044408734 +0000 UTC m=+469.639032777"
Jul 09 12:31:52.045598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.045580    2574 scope.go:117] "RemoveContainer" containerID="52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25"
Jul 09 12:31:52.046094 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.046036    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:52.046258 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.046231    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25\": container with ID starting with 52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25 not found: ID does not exist" containerID="52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25"
Jul 09 12:31:52.046313 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.046278    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25"} err="failed to get container status \"52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25\": rpc error: code = NotFound desc = could not find container \"52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25\": container with ID starting with 52106c4d7b35c942e686df94b4ccaaea165e7db77e2e9b73a355c7865a4cbf25 not found: ID does not exist"
Jul 09 12:31:52.047936 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.047915    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:52.049608 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.049589    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:52.051273 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.051253    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:31:52.087771 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.087720    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx" podStartSLOduration=2.087669087 podStartE2EDuration="2.087669087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:31:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:31:52.085181414 +0000 UTC m=+469.679805476" watchObservedRunningTime="2026-07-09 12:31:52.087669087 +0000 UTC m=+469.682293129"
Jul 09 12:31:52.971012 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.970980    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" path="/var/lib/kubelet/pods/2fc11349-796a-46f7-b2fa-76a9f1775933/volumes"
Jul 09 12:31:52.971236 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.971211    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j5gpk\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5gpk"
Jul 09 12:31:52.971358 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:31:52.971345    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b314636-f7e1-4f70-8cd8-402c4abe91cf" path="/var/lib/kubelet/pods/8b314636-f7e1-4f70-8cd8-402c4abe91cf/volumes"
Jul 09 12:31:52.973280 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:31:52.973259    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jgzbb\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="2fc11349-796a-46f7-b2fa-76a9f1775933" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jgzbb"
Jul 09 12:32:03.035279 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:03.035250    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dltzc"
Jul 09 12:32:03.035789 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:03.035330    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:32:23.814487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:23.814432    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"]
Jul 09 12:32:23.814949 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:23.814883    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx" podUID="378c0159-769a-4711-878f-a8c5978eea2c" containerName="manager" containerID="cri-o://2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d" gracePeriod=10
Jul 09 12:32:24.073731 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.073654    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:32:24.113866 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.113828    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/378c0159-769a-4711-878f-a8c5978eea2c-extensions-socket-volume\") pod \"378c0159-769a-4711-878f-a8c5978eea2c\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") "
Jul 09 12:32:24.114027 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.113885    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5v4l\" (UniqueName: \"kubernetes.io/projected/378c0159-769a-4711-878f-a8c5978eea2c-kube-api-access-j5v4l\") pod \"378c0159-769a-4711-878f-a8c5978eea2c\" (UID: \"378c0159-769a-4711-878f-a8c5978eea2c\") "
Jul 09 12:32:24.114260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.114236    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/378c0159-769a-4711-878f-a8c5978eea2c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "378c0159-769a-4711-878f-a8c5978eea2c" (UID: "378c0159-769a-4711-878f-a8c5978eea2c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:32:24.116053 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.116031    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/378c0159-769a-4711-878f-a8c5978eea2c-kube-api-access-j5v4l" (OuterVolumeSpecName: "kube-api-access-j5v4l") pod "378c0159-769a-4711-878f-a8c5978eea2c" (UID: "378c0159-769a-4711-878f-a8c5978eea2c"). InnerVolumeSpecName "kube-api-access-j5v4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:32:24.187666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.187624    2574 generic.go:356] "Generic (PLEG): container finished" podID="378c0159-769a-4711-878f-a8c5978eea2c" containerID="2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d" exitCode=0
Jul 09 12:32:24.187819 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.187695    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"
Jul 09 12:32:24.187819 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.187701    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx" event={"ID":"378c0159-769a-4711-878f-a8c5978eea2c","Type":"ContainerDied","Data":"2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d"}
Jul 09 12:32:24.187819 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.187745    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx" event={"ID":"378c0159-769a-4711-878f-a8c5978eea2c","Type":"ContainerDied","Data":"a20b8d56426a8cce132cc730b9e26119b5555fb33cabd3c17efaf62a91e4d575"}
Jul 09 12:32:24.187819 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.187767    2574 scope.go:117] "RemoveContainer" containerID="2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d"
Jul 09 12:32:24.197235 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.197218    2574 scope.go:117] "RemoveContainer" containerID="2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d"
Jul 09 12:32:24.197505 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:32:24.197490    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d\": container with ID starting with 2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d not found: ID does not exist" containerID="2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d"
Jul 09 12:32:24.197554 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.197513    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d"} err="failed to get container status \"2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d\": rpc error: code = NotFound desc = could not find container \"2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d\": container with ID starting with 2d209799a35f3ac28f6b8b91625e0861fd0f9cae81325409874e120334c3350d not found: ID does not exist"
Jul 09 12:32:24.211456 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.211419    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"]
Jul 09 12:32:24.214756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.214739    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/378c0159-769a-4711-878f-a8c5978eea2c-extensions-socket-volume\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:32:24.214831 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.214758    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5v4l\" (UniqueName: \"kubernetes.io/projected/378c0159-769a-4711-878f-a8c5978eea2c-kube-api-access-j5v4l\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:32:24.224696 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.224650    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k7rgx"]
Jul 09 12:32:24.970404 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:24.970369    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="378c0159-769a-4711-878f-a8c5978eea2c" path="/var/lib/kubelet/pods/378c0159-769a-4711-878f-a8c5978eea2c/volumes"
Jul 09 12:32:40.332951 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.330125    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"]
Jul 09 12:32:40.332951 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.330995    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378c0159-769a-4711-878f-a8c5978eea2c" containerName="manager"
Jul 09 12:32:40.332951 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.331014    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="378c0159-769a-4711-878f-a8c5978eea2c" containerName="manager"
Jul 09 12:32:40.332951 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.331180    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="378c0159-769a-4711-878f-a8c5978eea2c" containerName="manager"
Jul 09 12:32:40.334497 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.334471    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.337815 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.337789    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jslm8\""
Jul 09 12:32:40.357940 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.357909    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"]
Jul 09 12:32:40.451405 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934a66ff-914a-409a-911a-b8e31e3661f1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451480    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d289s\" (UniqueName: \"kubernetes.io/projected/934a66ff-914a-409a-911a-b8e31e3661f1-kube-api-access-d289s\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451942 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451674    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934a66ff-914a-409a-911a-b8e31e3661f1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451942 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451758    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451942 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934a66ff-914a-409a-911a-b8e31e3661f1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.451942 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.451856    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552513 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934a66ff-914a-409a-911a-b8e31e3661f1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552574    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d289s\" (UniqueName: \"kubernetes.io/projected/934a66ff-914a-409a-911a-b8e31e3661f1-kube-api-access-d289s\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552642    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934a66ff-914a-409a-911a-b8e31e3661f1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.552755 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552753    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934a66ff-914a-409a-911a-b8e31e3661f1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.553138 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.553138 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.552912    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.553138 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.553008    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.553138 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.553113    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.553390 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.553370    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934a66ff-914a-409a-911a-b8e31e3661f1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.553453 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.553435    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.554910 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.554888    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934a66ff-914a-409a-911a-b8e31e3661f1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.555540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.555519    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934a66ff-914a-409a-911a-b8e31e3661f1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.597192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.597109    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934a66ff-914a-409a-911a-b8e31e3661f1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.598263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.598239    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d289s\" (UniqueName: \"kubernetes.io/projected/934a66ff-914a-409a-911a-b8e31e3661f1-kube-api-access-d289s\") pod \"maas-default-gateway-openshift-default-56654dcf7-fw8tm\" (UID: \"934a66ff-914a-409a-911a-b8e31e3661f1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.648139 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.648097    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:40.795334 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.795305    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"]
Jul 09 12:32:40.797897 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:32:40.797868    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934a66ff_914a_409a_911a_b8e31e3661f1.slice/crio-e8ac2051bb3d3cc73ef6665231be2e508145577619a27a34421c2bfdc7673fa8 WatchSource:0}: Error finding container e8ac2051bb3d3cc73ef6665231be2e508145577619a27a34421c2bfdc7673fa8: Status 404 returned error can't find the container with id e8ac2051bb3d3cc73ef6665231be2e508145577619a27a34421c2bfdc7673fa8
Jul 09 12:32:40.799966 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.799933    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:32:40.800056 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.800001    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:32:40.800056 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:40.800032    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:32:41.280068 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:41.280032    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" event={"ID":"934a66ff-914a-409a-911a-b8e31e3661f1","Type":"ContainerStarted","Data":"8ad1765063d183c4c22c35a23124703178ba7cf30bb270635e6c50bf07f9ec0d"}
Jul 09 12:32:41.280262 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:41.280084    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" event={"ID":"934a66ff-914a-409a-911a-b8e31e3661f1","Type":"ContainerStarted","Data":"e8ac2051bb3d3cc73ef6665231be2e508145577619a27a34421c2bfdc7673fa8"}
Jul 09 12:32:41.310708 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:41.310643    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" podStartSLOduration=1.3106251150000001 podStartE2EDuration="1.310625115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:32:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:32:41.308820755 +0000 UTC m=+518.903444808" watchObservedRunningTime="2026-07-09 12:32:41.310625115 +0000 UTC m=+518.905249157"
Jul 09 12:32:41.649359 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:41.649255    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:41.654174 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:41.654147    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:42.285298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.285272    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:42.286293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.286270    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:32:42.515617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.515582    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:42.517885 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.517868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.520783 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.520759    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 12:32:42.520783 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.520769    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xjkhj\""
Jul 09 12:32:42.535574 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.535498    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:42.569403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.569363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml4wm\" (UniqueName: \"kubernetes.io/projected/7ed835bf-7078-4bf1-a36a-38937d54429f-kube-api-access-ml4wm\") pod \"limitador-limitador-64c94fd64f-dkljd\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.569571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.569441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7ed835bf-7078-4bf1-a36a-38937d54429f-config-file\") pod \"limitador-limitador-64c94fd64f-dkljd\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.613915 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.613884    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:42.670358 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.670318    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml4wm\" (UniqueName: \"kubernetes.io/projected/7ed835bf-7078-4bf1-a36a-38937d54429f-kube-api-access-ml4wm\") pod \"limitador-limitador-64c94fd64f-dkljd\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.670773 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.670374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7ed835bf-7078-4bf1-a36a-38937d54429f-config-file\") pod \"limitador-limitador-64c94fd64f-dkljd\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.671103 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.671080    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7ed835bf-7078-4bf1-a36a-38937d54429f-config-file\") pod \"limitador-limitador-64c94fd64f-dkljd\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.680858 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.680829    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml4wm\" (UniqueName: \"kubernetes.io/projected/7ed835bf-7078-4bf1-a36a-38937d54429f-kube-api-access-ml4wm\") pod \"limitador-limitador-64c94fd64f-dkljd\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:42.828075 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:42.827981    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:43.001268 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.001230    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:43.006036 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:32:43.006006    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed835bf_7078_4bf1_a36a_38937d54429f.slice/crio-7137904d0684828473c358d4133091aa964f0628e6707d79e8d2e598a4ed8b0d WatchSource:0}: Error finding container 7137904d0684828473c358d4133091aa964f0628e6707d79e8d2e598a4ed8b0d: Status 404 returned error can't find the container with id 7137904d0684828473c358d4133091aa964f0628e6707d79e8d2e598a4ed8b0d
Jul 09 12:32:43.290554 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.290506    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd" event={"ID":"7ed835bf-7078-4bf1-a36a-38937d54429f","Type":"ContainerStarted","Data":"7137904d0684828473c358d4133091aa964f0628e6707d79e8d2e598a4ed8b0d"}
Jul 09 12:32:43.453081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.453046    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-jbchj"]
Jul 09 12:32:43.456643 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.456617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:43.463926 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.463903    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h55pp\""
Jul 09 12:32:43.467092 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.467059    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jbchj"]
Jul 09 12:32:43.579576 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.579495    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtk48\" (UniqueName: \"kubernetes.io/projected/76c6ff49-339a-4396-9e18-0af7d180f8c8-kube-api-access-rtk48\") pod \"authorino-58cd955675-jbchj\" (UID: \"76c6ff49-339a-4396-9e18-0af7d180f8c8\") " pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:43.680591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.680537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtk48\" (UniqueName: \"kubernetes.io/projected/76c6ff49-339a-4396-9e18-0af7d180f8c8-kube-api-access-rtk48\") pod \"authorino-58cd955675-jbchj\" (UID: \"76c6ff49-339a-4396-9e18-0af7d180f8c8\") " pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:43.690850 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.690797    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtk48\" (UniqueName: \"kubernetes.io/projected/76c6ff49-339a-4396-9e18-0af7d180f8c8-kube-api-access-rtk48\") pod \"authorino-58cd955675-jbchj\" (UID: \"76c6ff49-339a-4396-9e18-0af7d180f8c8\") " pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:43.767260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.767218    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:43.980744 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:43.980716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jbchj"]
Jul 09 12:32:43.983892 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:32:43.983856    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76c6ff49_339a_4396_9e18_0af7d180f8c8.slice/crio-bef99201119569cd8ea9f78dfe4a2514916a8d6c6f960e1a298537b372228bb7 WatchSource:0}: Error finding container bef99201119569cd8ea9f78dfe4a2514916a8d6c6f960e1a298537b372228bb7: Status 404 returned error can't find the container with id bef99201119569cd8ea9f78dfe4a2514916a8d6c6f960e1a298537b372228bb7
Jul 09 12:32:44.297100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:44.297052    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jbchj" event={"ID":"76c6ff49-339a-4396-9e18-0af7d180f8c8","Type":"ContainerStarted","Data":"bef99201119569cd8ea9f78dfe4a2514916a8d6c6f960e1a298537b372228bb7"}
Jul 09 12:32:46.312545 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:46.312499    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd" event={"ID":"7ed835bf-7078-4bf1-a36a-38937d54429f","Type":"ContainerStarted","Data":"83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4"}
Jul 09 12:32:46.312975 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:46.312588    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:46.335433 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:46.335363    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd" podStartSLOduration=1.435445448 podStartE2EDuration="4.335342247s" totalImagesPullingTime="2.899896799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:32:46.333895084 +0000 UTC m=+523.928519167" watchObservedRunningTime="2026-07-09 12:32:46.335342247 +0000 UTC m=+523.929966291"
Jul 09 12:32:47.970175 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:47.970135    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jbchj"]
Jul 09 12:32:48.324603 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:48.324569    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jbchj" event={"ID":"76c6ff49-339a-4396-9e18-0af7d180f8c8","Type":"ContainerStarted","Data":"d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48"}
Jul 09 12:32:48.348293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:48.348234    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-jbchj" podStartSLOduration=1.555679136 podStartE2EDuration="5.348216054s" totalImagesPullingTime="3.792536918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:32:48.346391756 +0000 UTC m=+525.941015799" watchObservedRunningTime="2026-07-09 12:32:48.348216054 +0000 UTC m=+525.942840093"
Jul 09 12:32:49.329992 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:49.329950    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-jbchj" podUID="76c6ff49-339a-4396-9e18-0af7d180f8c8" containerName="authorino" containerID="cri-o://d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48" gracePeriod=30
Jul 09 12:32:49.573646 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:49.573620    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:49.639749 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:49.639602    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtk48\" (UniqueName: \"kubernetes.io/projected/76c6ff49-339a-4396-9e18-0af7d180f8c8-kube-api-access-rtk48\") pod \"76c6ff49-339a-4396-9e18-0af7d180f8c8\" (UID: \"76c6ff49-339a-4396-9e18-0af7d180f8c8\") "
Jul 09 12:32:49.641727 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:49.641698    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c6ff49-339a-4396-9e18-0af7d180f8c8-kube-api-access-rtk48" (OuterVolumeSpecName: "kube-api-access-rtk48") pod "76c6ff49-339a-4396-9e18-0af7d180f8c8" (UID: "76c6ff49-339a-4396-9e18-0af7d180f8c8"). InnerVolumeSpecName "kube-api-access-rtk48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:32:49.740493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:49.740455    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtk48\" (UniqueName: \"kubernetes.io/projected/76c6ff49-339a-4396-9e18-0af7d180f8c8-kube-api-access-rtk48\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:32:50.335887 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.335847    2574 generic.go:356] "Generic (PLEG): container finished" podID="76c6ff49-339a-4396-9e18-0af7d180f8c8" containerID="d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48" exitCode=0
Jul 09 12:32:50.336359 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.335907    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jbchj"
Jul 09 12:32:50.336359 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.335938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jbchj" event={"ID":"76c6ff49-339a-4396-9e18-0af7d180f8c8","Type":"ContainerDied","Data":"d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48"}
Jul 09 12:32:50.336359 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.335986    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jbchj" event={"ID":"76c6ff49-339a-4396-9e18-0af7d180f8c8","Type":"ContainerDied","Data":"bef99201119569cd8ea9f78dfe4a2514916a8d6c6f960e1a298537b372228bb7"}
Jul 09 12:32:50.336359 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.336013    2574 scope.go:117] "RemoveContainer" containerID="d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48"
Jul 09 12:32:50.344900 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.344884    2574 scope.go:117] "RemoveContainer" containerID="d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48"
Jul 09 12:32:50.345196 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:32:50.345177    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48\": container with ID starting with d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48 not found: ID does not exist" containerID="d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48"
Jul 09 12:32:50.345239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.345207    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48"} err="failed to get container status \"d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48\": rpc error: code = NotFound desc = could not find container \"d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48\": container with ID starting with d1a11bc9eb7753d141c86c97bc4203616fde658fb0b2fb606e2a0bfc6e6b3e48 not found: ID does not exist"
Jul 09 12:32:50.361243 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.361210    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jbchj"]
Jul 09 12:32:50.372138 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.372109    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-jbchj"]
Jul 09 12:32:50.970637 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:50.970603    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76c6ff49-339a-4396-9e18-0af7d180f8c8" path="/var/lib/kubelet/pods/76c6ff49-339a-4396-9e18-0af7d180f8c8/volumes"
Jul 09 12:32:57.320016 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:57.319981    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:57.622023 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:57.621930    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:57.622193 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:57.622119    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd" podUID="7ed835bf-7078-4bf1-a36a-38937d54429f" containerName="limitador" containerID="cri-o://83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4" gracePeriod=30
Jul 09 12:32:58.174996 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.174965    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:58.317055 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.317015    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml4wm\" (UniqueName: \"kubernetes.io/projected/7ed835bf-7078-4bf1-a36a-38937d54429f-kube-api-access-ml4wm\") pod \"7ed835bf-7078-4bf1-a36a-38937d54429f\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") "
Jul 09 12:32:58.317055 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.317055    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7ed835bf-7078-4bf1-a36a-38937d54429f-config-file\") pod \"7ed835bf-7078-4bf1-a36a-38937d54429f\" (UID: \"7ed835bf-7078-4bf1-a36a-38937d54429f\") "
Jul 09 12:32:58.317450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.317423    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ed835bf-7078-4bf1-a36a-38937d54429f-config-file" (OuterVolumeSpecName: "config-file") pod "7ed835bf-7078-4bf1-a36a-38937d54429f" (UID: "7ed835bf-7078-4bf1-a36a-38937d54429f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:32:58.319210 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.319188    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ed835bf-7078-4bf1-a36a-38937d54429f-kube-api-access-ml4wm" (OuterVolumeSpecName: "kube-api-access-ml4wm") pod "7ed835bf-7078-4bf1-a36a-38937d54429f" (UID: "7ed835bf-7078-4bf1-a36a-38937d54429f"). InnerVolumeSpecName "kube-api-access-ml4wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:32:58.377057 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.377023    2574 generic.go:356] "Generic (PLEG): container finished" podID="7ed835bf-7078-4bf1-a36a-38937d54429f" containerID="83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4" exitCode=0
Jul 09 12:32:58.377498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.377063    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd"
Jul 09 12:32:58.377498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.377107    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd" event={"ID":"7ed835bf-7078-4bf1-a36a-38937d54429f","Type":"ContainerDied","Data":"83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4"}
Jul 09 12:32:58.377498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.377157    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dkljd" event={"ID":"7ed835bf-7078-4bf1-a36a-38937d54429f","Type":"ContainerDied","Data":"7137904d0684828473c358d4133091aa964f0628e6707d79e8d2e598a4ed8b0d"}
Jul 09 12:32:58.377498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.377181    2574 scope.go:117] "RemoveContainer" containerID="83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4"
Jul 09 12:32:58.386190 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.386173    2574 scope.go:117] "RemoveContainer" containerID="83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4"
Jul 09 12:32:58.386434 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:32:58.386415    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4\": container with ID starting with 83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4 not found: ID does not exist" containerID="83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4"
Jul 09 12:32:58.386479 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.386443    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4"} err="failed to get container status \"83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4\": rpc error: code = NotFound desc = could not find container \"83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4\": container with ID starting with 83445e618fceea147b3b204b5a7d3a80735e8d6f98976d01fb520b79c6b76ad4 not found: ID does not exist"
Jul 09 12:32:58.403943 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.403887    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:58.407421 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.407399    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dkljd"]
Jul 09 12:32:58.418123 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.418098    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml4wm\" (UniqueName: \"kubernetes.io/projected/7ed835bf-7078-4bf1-a36a-38937d54429f-kube-api-access-ml4wm\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:32:58.418123 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.418123    2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7ed835bf-7078-4bf1-a36a-38937d54429f-config-file\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:32:58.970665 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:32:58.970627    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ed835bf-7078-4bf1-a36a-38937d54429f" path="/var/lib/kubelet/pods/7ed835bf-7078-4bf1-a36a-38937d54429f/volumes"
Jul 09 12:33:04.322162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322128    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"]
Jul 09 12:33:04.322553 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322498    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c6ff49-339a-4396-9e18-0af7d180f8c8" containerName="authorino"
Jul 09 12:33:04.322553 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322509    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c6ff49-339a-4396-9e18-0af7d180f8c8" containerName="authorino"
Jul 09 12:33:04.322553 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322525    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed835bf-7078-4bf1-a36a-38937d54429f" containerName="limitador"
Jul 09 12:33:04.322553 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322531    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed835bf-7078-4bf1-a36a-38937d54429f" containerName="limitador"
Jul 09 12:33:04.322710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322584    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed835bf-7078-4bf1-a36a-38937d54429f" containerName="limitador"
Jul 09 12:33:04.322710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.322591    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c6ff49-339a-4396-9e18-0af7d180f8c8" containerName="authorino"
Jul 09 12:33:04.325158 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.325132    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.327936 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.327909    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 09 12:33:04.327936 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.327929    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 12:33:04.328074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.328040    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 12:33:04.328147 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.328133    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-rbcxh\""
Jul 09 12:33:04.338172 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.338144    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"]
Jul 09 12:33:04.472173 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.472135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fe0bb06b-018c-4554-b6b8-056a545b72d6-data\") pod \"postgres-69fb4594cf-ds52m\" (UID: \"fe0bb06b-018c-4554-b6b8-056a545b72d6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.472325 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.472237    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbxh7\" (UniqueName: \"kubernetes.io/projected/fe0bb06b-018c-4554-b6b8-056a545b72d6-kube-api-access-qbxh7\") pod \"postgres-69fb4594cf-ds52m\" (UID: \"fe0bb06b-018c-4554-b6b8-056a545b72d6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.573124 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.573026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fe0bb06b-018c-4554-b6b8-056a545b72d6-data\") pod \"postgres-69fb4594cf-ds52m\" (UID: \"fe0bb06b-018c-4554-b6b8-056a545b72d6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.573124 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.573099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbxh7\" (UniqueName: \"kubernetes.io/projected/fe0bb06b-018c-4554-b6b8-056a545b72d6-kube-api-access-qbxh7\") pod \"postgres-69fb4594cf-ds52m\" (UID: \"fe0bb06b-018c-4554-b6b8-056a545b72d6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.573437 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.573417    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fe0bb06b-018c-4554-b6b8-056a545b72d6-data\") pod \"postgres-69fb4594cf-ds52m\" (UID: \"fe0bb06b-018c-4554-b6b8-056a545b72d6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.583048 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.583011    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbxh7\" (UniqueName: \"kubernetes.io/projected/fe0bb06b-018c-4554-b6b8-056a545b72d6-kube-api-access-qbxh7\") pod \"postgres-69fb4594cf-ds52m\" (UID: \"fe0bb06b-018c-4554-b6b8-056a545b72d6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.635641 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.635594    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:04.775587 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:04.775555    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"]
Jul 09 12:33:04.778200 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:33:04.778175    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0bb06b_018c_4554_b6b8_056a545b72d6.slice/crio-61276eadfc3c86dedc32bb6a77e41e0c6e540f3237acce7228b0edde17bf62d0 WatchSource:0}: Error finding container 61276eadfc3c86dedc32bb6a77e41e0c6e540f3237acce7228b0edde17bf62d0: Status 404 returned error can't find the container with id 61276eadfc3c86dedc32bb6a77e41e0c6e540f3237acce7228b0edde17bf62d0
Jul 09 12:33:05.414987 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:05.414950    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m" event={"ID":"fe0bb06b-018c-4554-b6b8-056a545b72d6","Type":"ContainerStarted","Data":"61276eadfc3c86dedc32bb6a77e41e0c6e540f3237acce7228b0edde17bf62d0"}
Jul 09 12:33:10.442880 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:10.442836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m" event={"ID":"fe0bb06b-018c-4554-b6b8-056a545b72d6","Type":"ContainerStarted","Data":"ecb81b69d2d9f091d077497e11ad5d214962c0ef4d8da9ae7e0844bbad26a702"}
Jul 09 12:33:10.443285 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:10.442932    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:10.474168 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:10.474115    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m" podStartSLOduration=1.767695402 podStartE2EDuration="6.474097877s" totalImagesPullingTime="4.706402475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:33:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:33:10.47355202 +0000 UTC m=+548.068176090" watchObservedRunningTime="2026-07-09 12:33:10.474097877 +0000 UTC m=+548.068721920"
Jul 09 12:33:16.477464 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:16.477420    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ds52m"
Jul 09 12:33:17.307188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.307156    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s62fk"]
Jul 09 12:33:17.312243 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.312219    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:17.316332 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.316308    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h55pp\""
Jul 09 12:33:17.388496 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.388460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf8hk\" (UniqueName: \"kubernetes.io/projected/7fd7b588-3f2a-4a32-959b-a6292024706a-kube-api-access-pf8hk\") pod \"authorino-5d4b67c446-s62fk\" (UID: \"7fd7b588-3f2a-4a32-959b-a6292024706a\") " pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:17.416218 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.416185    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s62fk"]
Jul 09 12:33:17.467545 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.467511    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s62fk"]
Jul 09 12:33:17.467828 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:33:17.467807    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pf8hk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-s62fk" podUID="7fd7b588-3f2a-4a32-959b-a6292024706a"
Jul 09 12:33:17.476625 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.476600    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:17.482345 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.482322    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:17.489427 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.489407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf8hk\" (UniqueName: \"kubernetes.io/projected/7fd7b588-3f2a-4a32-959b-a6292024706a-kube-api-access-pf8hk\") pod \"authorino-5d4b67c446-s62fk\" (UID: \"7fd7b588-3f2a-4a32-959b-a6292024706a\") " pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:17.521766 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.521729    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf8hk\" (UniqueName: \"kubernetes.io/projected/7fd7b588-3f2a-4a32-959b-a6292024706a-kube-api-access-pf8hk\") pod \"authorino-5d4b67c446-s62fk\" (UID: \"7fd7b588-3f2a-4a32-959b-a6292024706a\") " pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:17.547143 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.547108    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-688c5db7dd-7bf4g"]
Jul 09 12:33:17.549371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.549354    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:17.564888 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.564810    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-688c5db7dd-7bf4g"]
Jul 09 12:33:17.590797 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.590766    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf8hk\" (UniqueName: \"kubernetes.io/projected/7fd7b588-3f2a-4a32-959b-a6292024706a-kube-api-access-pf8hk\") pod \"7fd7b588-3f2a-4a32-959b-a6292024706a\" (UID: \"7fd7b588-3f2a-4a32-959b-a6292024706a\") "
Jul 09 12:33:17.590991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.590944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nhzq\" (UniqueName: \"kubernetes.io/projected/ffeb01e3-5798-4f6b-8880-36ed46d423e5-kube-api-access-2nhzq\") pod \"authorino-688c5db7dd-7bf4g\" (UID: \"ffeb01e3-5798-4f6b-8880-36ed46d423e5\") " pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:17.593172 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.593135    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd7b588-3f2a-4a32-959b-a6292024706a-kube-api-access-pf8hk" (OuterVolumeSpecName: "kube-api-access-pf8hk") pod "7fd7b588-3f2a-4a32-959b-a6292024706a" (UID: "7fd7b588-3f2a-4a32-959b-a6292024706a"). InnerVolumeSpecName "kube-api-access-pf8hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:33:17.692338 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.692299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nhzq\" (UniqueName: \"kubernetes.io/projected/ffeb01e3-5798-4f6b-8880-36ed46d423e5-kube-api-access-2nhzq\") pod \"authorino-688c5db7dd-7bf4g\" (UID: \"ffeb01e3-5798-4f6b-8880-36ed46d423e5\") " pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:17.692560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.692369    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf8hk\" (UniqueName: \"kubernetes.io/projected/7fd7b588-3f2a-4a32-959b-a6292024706a-kube-api-access-pf8hk\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:33:17.702634 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.702605    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nhzq\" (UniqueName: \"kubernetes.io/projected/ffeb01e3-5798-4f6b-8880-36ed46d423e5-kube-api-access-2nhzq\") pod \"authorino-688c5db7dd-7bf4g\" (UID: \"ffeb01e3-5798-4f6b-8880-36ed46d423e5\") " pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:17.769658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.769620    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-688c5db7dd-7bf4g"]
Jul 09 12:33:17.769955 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.769932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:17.834709 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.834653    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bb55c7cb8-6xrb5"]
Jul 09 12:33:17.837756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.837730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:17.842418 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.842392    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 12:33:17.856997 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.856962    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bb55c7cb8-6xrb5"]
Jul 09 12:33:17.894392 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.894360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5772w\" (UniqueName: \"kubernetes.io/projected/4618d08b-a24e-4428-a98c-2179142cb880-kube-api-access-5772w\") pod \"authorino-6bb55c7cb8-6xrb5\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") " pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:17.894549 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.894410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4618d08b-a24e-4428-a98c-2179142cb880-tls-cert\") pod \"authorino-6bb55c7cb8-6xrb5\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") " pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:17.924555 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.924530    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-688c5db7dd-7bf4g"]
Jul 09 12:33:17.927024 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:33:17.926995    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffeb01e3_5798_4f6b_8880_36ed46d423e5.slice/crio-f6ea5ab22e3d1fcc4afcb063c97ef6c1f2755701a0010f205a6767f232359281 WatchSource:0}: Error finding container f6ea5ab22e3d1fcc4afcb063c97ef6c1f2755701a0010f205a6767f232359281: Status 404 returned error can't find the container with id f6ea5ab22e3d1fcc4afcb063c97ef6c1f2755701a0010f205a6767f232359281
Jul 09 12:33:17.995274 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.995230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4618d08b-a24e-4428-a98c-2179142cb880-tls-cert\") pod \"authorino-6bb55c7cb8-6xrb5\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") " pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:17.995449 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.995328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5772w\" (UniqueName: \"kubernetes.io/projected/4618d08b-a24e-4428-a98c-2179142cb880-kube-api-access-5772w\") pod \"authorino-6bb55c7cb8-6xrb5\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") " pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:17.998162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:17.998127    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4618d08b-a24e-4428-a98c-2179142cb880-tls-cert\") pod \"authorino-6bb55c7cb8-6xrb5\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") " pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:18.004812 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.004790    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5772w\" (UniqueName: \"kubernetes.io/projected/4618d08b-a24e-4428-a98c-2179142cb880-kube-api-access-5772w\") pod \"authorino-6bb55c7cb8-6xrb5\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") " pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:18.147736 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.147621    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:33:18.287911 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.287883    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bb55c7cb8-6xrb5"]
Jul 09 12:33:18.483353 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.483313    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688c5db7dd-7bf4g" event={"ID":"ffeb01e3-5798-4f6b-8880-36ed46d423e5","Type":"ContainerStarted","Data":"f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368"}
Jul 09 12:33:18.483836 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.483332    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-688c5db7dd-7bf4g" podUID="ffeb01e3-5798-4f6b-8880-36ed46d423e5" containerName="authorino" containerID="cri-o://f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368" gracePeriod=30
Jul 09 12:33:18.483836 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.483366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688c5db7dd-7bf4g" event={"ID":"ffeb01e3-5798-4f6b-8880-36ed46d423e5","Type":"ContainerStarted","Data":"f6ea5ab22e3d1fcc4afcb063c97ef6c1f2755701a0010f205a6767f232359281"}
Jul 09 12:33:18.484764 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.484740    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5" event={"ID":"4618d08b-a24e-4428-a98c-2179142cb880","Type":"ContainerStarted","Data":"49839c92538026b0ce71e91aa4f51ff49f3c38cf69e1ed7e23c72e03fb70afe9"}
Jul 09 12:33:18.484870 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.484782    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s62fk"
Jul 09 12:33:18.514408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.514356    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-688c5db7dd-7bf4g" podStartSLOduration=1.024885345 podStartE2EDuration="1.514341487s" totalImagesPullingTime="489.456142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:33:18.512555673 +0000 UTC m=+556.107179735" watchObservedRunningTime="2026-07-09 12:33:18.514341487 +0000 UTC m=+556.108965529"
Jul 09 12:33:18.566190 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.566156    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s62fk"]
Jul 09 12:33:18.572739 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.572709    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s62fk"]
Jul 09 12:33:18.747510 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.747485    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:18.801946 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.801916    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nhzq\" (UniqueName: \"kubernetes.io/projected/ffeb01e3-5798-4f6b-8880-36ed46d423e5-kube-api-access-2nhzq\") pod \"ffeb01e3-5798-4f6b-8880-36ed46d423e5\" (UID: \"ffeb01e3-5798-4f6b-8880-36ed46d423e5\") "
Jul 09 12:33:18.803982 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.803954    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffeb01e3-5798-4f6b-8880-36ed46d423e5-kube-api-access-2nhzq" (OuterVolumeSpecName: "kube-api-access-2nhzq") pod "ffeb01e3-5798-4f6b-8880-36ed46d423e5" (UID: "ffeb01e3-5798-4f6b-8880-36ed46d423e5"). InnerVolumeSpecName "kube-api-access-2nhzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:33:18.903532 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.903498    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nhzq\" (UniqueName: \"kubernetes.io/projected/ffeb01e3-5798-4f6b-8880-36ed46d423e5-kube-api-access-2nhzq\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:33:18.971627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:18.971595    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fd7b588-3f2a-4a32-959b-a6292024706a" path="/var/lib/kubelet/pods/7fd7b588-3f2a-4a32-959b-a6292024706a/volumes"
Jul 09 12:33:19.491322 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.491289    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5" event={"ID":"4618d08b-a24e-4428-a98c-2179142cb880","Type":"ContainerStarted","Data":"09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63"}
Jul 09 12:33:19.492556 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.492529    2574 generic.go:356] "Generic (PLEG): container finished" podID="ffeb01e3-5798-4f6b-8880-36ed46d423e5" containerID="f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368" exitCode=2
Jul 09 12:33:19.492912 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.492563    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688c5db7dd-7bf4g"
Jul 09 12:33:19.492912 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.492580    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688c5db7dd-7bf4g" event={"ID":"ffeb01e3-5798-4f6b-8880-36ed46d423e5","Type":"ContainerDied","Data":"f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368"}
Jul 09 12:33:19.492912 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.492606    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688c5db7dd-7bf4g" event={"ID":"ffeb01e3-5798-4f6b-8880-36ed46d423e5","Type":"ContainerDied","Data":"f6ea5ab22e3d1fcc4afcb063c97ef6c1f2755701a0010f205a6767f232359281"}
Jul 09 12:33:19.492912 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.492622    2574 scope.go:117] "RemoveContainer" containerID="f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368"
Jul 09 12:33:19.502641 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.502610    2574 scope.go:117] "RemoveContainer" containerID="f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368"
Jul 09 12:33:19.502944 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:33:19.502922    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368\": container with ID starting with f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368 not found: ID does not exist" containerID="f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368"
Jul 09 12:33:19.503009 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.502952    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368"} err="failed to get container status \"f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368\": rpc error: code = NotFound desc = could not find container \"f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368\": container with ID starting with f0c0c6077b9d46d85187f74aa10ee08cef411e9478c8b644ec1da1e5789d7368 not found: ID does not exist"
Jul 09 12:33:19.551808 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.551757    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5" podStartSLOduration=2.093789628 podStartE2EDuration="2.551739759s" totalImagesPullingTime="457.950131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:33:19.526508964 +0000 UTC m=+557.121133008" watchObservedRunningTime="2026-07-09 12:33:19.551739759 +0000 UTC m=+557.146363803"
Jul 09 12:33:19.553076 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.553057    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-688c5db7dd-7bf4g"]
Jul 09 12:33:19.555770 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:19.555746    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-688c5db7dd-7bf4g"]
Jul 09 12:33:20.970701 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:20.970648    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffeb01e3-5798-4f6b-8880-36ed46d423e5" path="/var/lib/kubelet/pods/ffeb01e3-5798-4f6b-8880-36ed46d423e5/volumes"
Jul 09 12:33:59.121242 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.121202    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"]
Jul 09 12:33:59.121834 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.121562    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffeb01e3-5798-4f6b-8880-36ed46d423e5" containerName="authorino"
Jul 09 12:33:59.121834 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.121573    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffeb01e3-5798-4f6b-8880-36ed46d423e5" containerName="authorino"
Jul 09 12:33:59.121834 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.121666    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffeb01e3-5798-4f6b-8880-36ed46d423e5" containerName="authorino"
Jul 09 12:33:59.124720 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.124691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.127294 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.127277    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 12:33:59.127836 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.127820    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k6plx\""
Jul 09 12:33:59.142495 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.142466    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"]
Jul 09 12:33:59.174157 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.174121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-openshift-service-ca\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.174335 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.174182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-maas-api-tls\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.174335 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.174211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv8bz\" (UniqueName: \"kubernetes.io/projected/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-kube-api-access-dv8bz\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.274586 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.274545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-openshift-service-ca\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.274800 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.274598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-maas-api-tls\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.274800 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.274624    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv8bz\" (UniqueName: \"kubernetes.io/projected/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-kube-api-access-dv8bz\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.275414 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.275394    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-openshift-service-ca\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.277364 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.277340    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-maas-api-tls\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.283920 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.283895    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv8bz\" (UniqueName: \"kubernetes.io/projected/47a77dd0-b266-4244-8a84-5d4e6bba2f4b-kube-api-access-dv8bz\") pod \"maas-api-74cf5c9f5d-fgccq\" (UID: \"47a77dd0-b266-4244-8a84-5d4e6bba2f4b\") " pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.436161 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.436070    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:33:59.593908 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.593736    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"]
Jul 09 12:33:59.596833 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:33:59.596801    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a77dd0_b266_4244_8a84_5d4e6bba2f4b.slice/crio-25ab0c5c58077d4d31afc2efbbfd9e31c6d18506350e43165baecb94be065fc1 WatchSource:0}: Error finding container 25ab0c5c58077d4d31afc2efbbfd9e31c6d18506350e43165baecb94be065fc1: Status 404 returned error can't find the container with id 25ab0c5c58077d4d31afc2efbbfd9e31c6d18506350e43165baecb94be065fc1
Jul 09 12:33:59.685761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:33:59.685671    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq" event={"ID":"47a77dd0-b266-4244-8a84-5d4e6bba2f4b","Type":"ContainerStarted","Data":"25ab0c5c58077d4d31afc2efbbfd9e31c6d18506350e43165baecb94be065fc1"}
Jul 09 12:34:01.698429 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:01.698392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq" event={"ID":"47a77dd0-b266-4244-8a84-5d4e6bba2f4b","Type":"ContainerStarted","Data":"6acd9be631867cefd2ca8fe00f5968fff1a542e7070f76fc6c7bf10e3b130fe4"}
Jul 09 12:34:01.698841 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:01.698530    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:34:01.727803 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:01.727744    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq" podStartSLOduration=0.807625095 podStartE2EDuration="2.727721982s" totalImagesPullingTime="1.920096887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:34:01.726757986 +0000 UTC m=+599.321382032" watchObservedRunningTime="2026-07-09 12:34:01.727721982 +0000 UTC m=+599.322346026"
Jul 09 12:34:02.888217 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:02.888184    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:34:02.890575 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:02.890542    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:34:07.709162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.709128    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74cf5c9f5d-fgccq"
Jul 09 12:34:07.730126 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.730094    2574 generic.go:356] "Generic (PLEG): container finished" podID="934a66ff-914a-409a-911a-b8e31e3661f1" containerID="8ad1765063d183c4c22c35a23124703178ba7cf30bb270635e6c50bf07f9ec0d" exitCode=0
Jul 09 12:34:07.730292 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.730167    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" event={"ID":"934a66ff-914a-409a-911a-b8e31e3661f1","Type":"ContainerDied","Data":"8ad1765063d183c4c22c35a23124703178ba7cf30bb270635e6c50bf07f9ec0d"}
Jul 09 12:34:07.730538 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.730525    2574 scope.go:117] "RemoveContainer" containerID="8ad1765063d183c4c22c35a23124703178ba7cf30bb270635e6c50bf07f9ec0d"
Jul 09 12:34:07.731624 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.731587    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:34:07.731759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.731675    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:34:07.731759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:07.731735    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:34:08.738651 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:08.738606    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" event={"ID":"934a66ff-914a-409a-911a-b8e31e3661f1","Type":"ContainerStarted","Data":"78a8392f2d7d071b1c98944463187826dad39c143820221dedef9de52c2ac0d7"}
Jul 09 12:34:09.649085 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:09.649040    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:34:09.650003 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:09.649969    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fw8tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 09 12:34:09.650140 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:09.650064    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" podUID="934a66ff-914a-409a-911a-b8e31e3661f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 09 12:34:10.649412 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:10.649358    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fw8tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 09 12:34:10.649970 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:10.649434    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" podUID="934a66ff-914a-409a-911a-b8e31e3661f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 09 12:34:11.649188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:11.649147    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fw8tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 09 12:34:11.649527 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:11.649493    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" podUID="934a66ff-914a-409a-911a-b8e31e3661f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 09 12:34:12.285969 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:12.285926    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:34:12.649369 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:12.649270    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fw8tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 09 12:34:12.649369 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:12.649333    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm" podUID="934a66ff-914a-409a-911a-b8e31e3661f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 09 12:34:13.653231 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:13.653194    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:34:13.654081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:13.654065    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fw8tm"
Jul 09 12:34:21.614633 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.614601    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"]
Jul 09 12:34:21.621162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.621142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.624751 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.624726    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w26c9\""
Jul 09 12:34:21.624869 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.624781    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 09 12:34:21.624869 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.624729    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 12:34:21.624869 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.624730    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 12:34:21.629438 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.629414    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"]
Jul 09 12:34:21.666449 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.666413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.666648 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.666462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdg72\" (UniqueName: \"kubernetes.io/projected/b6f7b374-00ee-416d-bf8e-82e38c3b198e-kube-api-access-hdg72\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.666648 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.666483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.666648 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.666578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f7b374-00ee-416d-bf8e-82e38c3b198e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.666648 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.666622    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.666861 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.666705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.767534 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.767488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.767799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.767552    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.767799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.767739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.767799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.767783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdg72\" (UniqueName: \"kubernetes.io/projected/b6f7b374-00ee-416d-bf8e-82e38c3b198e-kube-api-access-hdg72\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.767983 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.767812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.767983 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.767861    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f7b374-00ee-416d-bf8e-82e38c3b198e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.768086 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.768005    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.768298 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.768259    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.768462 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.768440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.770903 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.770871    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f7b374-00ee-416d-bf8e-82e38c3b198e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.771135 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.771117    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f7b374-00ee-416d-bf8e-82e38c3b198e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.776783 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.776758    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdg72\" (UniqueName: \"kubernetes.io/projected/b6f7b374-00ee-416d-bf8e-82e38c3b198e-kube-api-access-hdg72\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2\" (UID: \"b6f7b374-00ee-416d-bf8e-82e38c3b198e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:21.932174 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:21.932082    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:22.078528 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:22.078503    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"]
Jul 09 12:34:22.081129 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:34:22.081100    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f7b374_00ee_416d_bf8e_82e38c3b198e.slice/crio-652ed4ad1784acf5253db17bcc9e0d9c033bd265c67b8b7078e35723f479de0b WatchSource:0}: Error finding container 652ed4ad1784acf5253db17bcc9e0d9c033bd265c67b8b7078e35723f479de0b: Status 404 returned error can't find the container with id 652ed4ad1784acf5253db17bcc9e0d9c033bd265c67b8b7078e35723f479de0b
Jul 09 12:34:22.082869 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:22.082844    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:34:22.829079 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:22.829039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2" event={"ID":"b6f7b374-00ee-416d-bf8e-82e38c3b198e","Type":"ContainerStarted","Data":"652ed4ad1784acf5253db17bcc9e0d9c033bd265c67b8b7078e35723f479de0b"}
Jul 09 12:34:23.836284 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:23.836249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2" event={"ID":"b6f7b374-00ee-416d-bf8e-82e38c3b198e","Type":"ContainerStarted","Data":"ca37896a8b157f0df79a30eac8f7906b820bb7d235e0c24afdc9b7b29a1fb96c"}
Jul 09 12:34:23.860351 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:23.860294    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2" podStartSLOduration=1.74432617 podStartE2EDuration="2.860277924s" totalImagesPullingTime="1.115951754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:34:23.858262811 +0000 UTC m=+621.452886853" watchObservedRunningTime="2026-07-09 12:34:23.860277924 +0000 UTC m=+621.454901966"
Jul 09 12:34:31.932832 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:31.932743    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:31.932832 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:31.932834    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:31.949292 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:31.949256    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:32.893352 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:32.893319    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2"
Jul 09 12:34:38.597728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.597694    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"]
Jul 09 12:34:38.601259 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.601238    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.604424 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.604403    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 09 12:34:38.616434 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.616406    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"]
Jul 09 12:34:38.716236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.716202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.716399 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.716238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.716399 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.716348    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.716477 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.716402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1758a160-9fc0-4bf5-9a46-815a133db5bb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.716511 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.716471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rkmr\" (UniqueName: \"kubernetes.io/projected/1758a160-9fc0-4bf5-9a46-815a133db5bb-kube-api-access-7rkmr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.716547 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.716510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.817861 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.817811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.817895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.817922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.817974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.818018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1758a160-9fc0-4bf5-9a46-815a133db5bb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818318 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.818168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rkmr\" (UniqueName: \"kubernetes.io/projected/1758a160-9fc0-4bf5-9a46-815a133db5bb-kube-api-access-7rkmr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818318 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.818262    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818422 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.818349    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.818422 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.818397    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.820246 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.820224    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1758a160-9fc0-4bf5-9a46-815a133db5bb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.820445 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.820422    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1758a160-9fc0-4bf5-9a46-815a133db5bb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.828025 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.827992    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rkmr\" (UniqueName: \"kubernetes.io/projected/1758a160-9fc0-4bf5-9a46-815a133db5bb-kube-api-access-7rkmr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t7mjt\" (UID: \"1758a160-9fc0-4bf5-9a46-815a133db5bb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:38.911341 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:38.911243    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:39.075316 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:39.075291    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"]
Jul 09 12:34:39.077337 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:34:39.077303    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1758a160_9fc0_4bf5_9a46_815a133db5bb.slice/crio-77b133f7df534534f166704b470f9dc17ced6e5c4697f94d9a2a81af08961e66 WatchSource:0}: Error finding container 77b133f7df534534f166704b470f9dc17ced6e5c4697f94d9a2a81af08961e66: Status 404 returned error can't find the container with id 77b133f7df534534f166704b470f9dc17ced6e5c4697f94d9a2a81af08961e66
Jul 09 12:34:39.919023 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:39.918973    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt" event={"ID":"1758a160-9fc0-4bf5-9a46-815a133db5bb","Type":"ContainerStarted","Data":"4bf74366f9926f5fd37913bd9d9377ca8aa5518e265ec366e804e4f1acfa6d15"}
Jul 09 12:34:39.919494 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:39.919040    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt" event={"ID":"1758a160-9fc0-4bf5-9a46-815a133db5bb","Type":"ContainerStarted","Data":"77b133f7df534534f166704b470f9dc17ced6e5c4697f94d9a2a81af08961e66"}
Jul 09 12:34:39.941822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:39.941765    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt" podStartSLOduration=1.7507476789999998 podStartE2EDuration="1.941748985s" totalImagesPullingTime="191.001306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:34:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:34:39.940127865 +0000 UTC m=+637.534751928" watchObservedRunningTime="2026-07-09 12:34:39.941748985 +0000 UTC m=+637.536373028"
Jul 09 12:34:48.911920 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:48.911878    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:48.912538 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:48.911969    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:48.924439 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:48.924409    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:34:48.979777 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:34:48.979745    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t7mjt"
Jul 09 12:36:15.260224 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.260182    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c88d8b957-f6f5m"]
Jul 09 12:36:15.263902 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.263877    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.272932 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.272907    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c88d8b957-f6f5m"]
Jul 09 12:36:15.396231 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.396195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg5mv\" (UniqueName: \"kubernetes.io/projected/53ed4081-acd3-4d47-9783-d807e86b36b7-kube-api-access-bg5mv\") pod \"authorino-5c88d8b957-f6f5m\" (UID: \"53ed4081-acd3-4d47-9783-d807e86b36b7\") " pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.396401 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.396250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53ed4081-acd3-4d47-9783-d807e86b36b7-tls-cert\") pod \"authorino-5c88d8b957-f6f5m\" (UID: \"53ed4081-acd3-4d47-9783-d807e86b36b7\") " pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.497603 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.497562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg5mv\" (UniqueName: \"kubernetes.io/projected/53ed4081-acd3-4d47-9783-d807e86b36b7-kube-api-access-bg5mv\") pod \"authorino-5c88d8b957-f6f5m\" (UID: \"53ed4081-acd3-4d47-9783-d807e86b36b7\") " pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.497844 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.497645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53ed4081-acd3-4d47-9783-d807e86b36b7-tls-cert\") pod \"authorino-5c88d8b957-f6f5m\" (UID: \"53ed4081-acd3-4d47-9783-d807e86b36b7\") " pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.500055 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.500030    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53ed4081-acd3-4d47-9783-d807e86b36b7-tls-cert\") pod \"authorino-5c88d8b957-f6f5m\" (UID: \"53ed4081-acd3-4d47-9783-d807e86b36b7\") " pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.506336 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.506317    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg5mv\" (UniqueName: \"kubernetes.io/projected/53ed4081-acd3-4d47-9783-d807e86b36b7-kube-api-access-bg5mv\") pod \"authorino-5c88d8b957-f6f5m\" (UID: \"53ed4081-acd3-4d47-9783-d807e86b36b7\") " pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.574077 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.573979    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c88d8b957-f6f5m"
Jul 09 12:36:15.713941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:15.713915    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c88d8b957-f6f5m"]
Jul 09 12:36:15.716782 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:36:15.716753    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53ed4081_acd3_4d47_9783_d807e86b36b7.slice/crio-13d613f85745a2169235c599d2ab96082040383a5fbb5a0f5d1b7818a1d4218d WatchSource:0}: Error finding container 13d613f85745a2169235c599d2ab96082040383a5fbb5a0f5d1b7818a1d4218d: Status 404 returned error can't find the container with id 13d613f85745a2169235c599d2ab96082040383a5fbb5a0f5d1b7818a1d4218d
Jul 09 12:36:16.426834 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.426741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c88d8b957-f6f5m" event={"ID":"53ed4081-acd3-4d47-9783-d807e86b36b7","Type":"ContainerStarted","Data":"c370bab14618093b2fa347d072f6869031f9a360105fe3d42a4185d17f47054e"}
Jul 09 12:36:16.426834 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.426798    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c88d8b957-f6f5m" event={"ID":"53ed4081-acd3-4d47-9783-d807e86b36b7","Type":"ContainerStarted","Data":"13d613f85745a2169235c599d2ab96082040383a5fbb5a0f5d1b7818a1d4218d"}
Jul 09 12:36:16.447241 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.447186    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c88d8b957-f6f5m" podStartSLOduration=1.029461022 podStartE2EDuration="1.447169574s" totalImagesPullingTime="417.708552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:36:16.444450374 +0000 UTC m=+734.039074417" watchObservedRunningTime="2026-07-09 12:36:16.447169574 +0000 UTC m=+734.041793616"
Jul 09 12:36:16.491311 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.491271    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bb55c7cb8-6xrb5"]
Jul 09 12:36:16.491613 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.491581    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5" podUID="4618d08b-a24e-4428-a98c-2179142cb880" containerName="authorino" containerID="cri-o://09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63" gracePeriod=30
Jul 09 12:36:16.763988 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.763959    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:36:16.811980 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.811791    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5772w\" (UniqueName: \"kubernetes.io/projected/4618d08b-a24e-4428-a98c-2179142cb880-kube-api-access-5772w\") pod \"4618d08b-a24e-4428-a98c-2179142cb880\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") "
Jul 09 12:36:16.811980 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.811964    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4618d08b-a24e-4428-a98c-2179142cb880-tls-cert\") pod \"4618d08b-a24e-4428-a98c-2179142cb880\" (UID: \"4618d08b-a24e-4428-a98c-2179142cb880\") "
Jul 09 12:36:16.815090 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.815031    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4618d08b-a24e-4428-a98c-2179142cb880-kube-api-access-5772w" (OuterVolumeSpecName: "kube-api-access-5772w") pod "4618d08b-a24e-4428-a98c-2179142cb880" (UID: "4618d08b-a24e-4428-a98c-2179142cb880"). InnerVolumeSpecName "kube-api-access-5772w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:36:16.824772 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.824670    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4618d08b-a24e-4428-a98c-2179142cb880-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4618d08b-a24e-4428-a98c-2179142cb880" (UID: "4618d08b-a24e-4428-a98c-2179142cb880"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:36:16.913057 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.913027    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5772w\" (UniqueName: \"kubernetes.io/projected/4618d08b-a24e-4428-a98c-2179142cb880-kube-api-access-5772w\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:36:16.913057 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:16.913055    2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4618d08b-a24e-4428-a98c-2179142cb880-tls-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:36:17.433306 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.433269    2574 generic.go:356] "Generic (PLEG): container finished" podID="4618d08b-a24e-4428-a98c-2179142cb880" containerID="09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63" exitCode=0
Jul 09 12:36:17.433785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.433347    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5"
Jul 09 12:36:17.433785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.433354    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5" event={"ID":"4618d08b-a24e-4428-a98c-2179142cb880","Type":"ContainerDied","Data":"09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63"}
Jul 09 12:36:17.433785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.433396    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bb55c7cb8-6xrb5" event={"ID":"4618d08b-a24e-4428-a98c-2179142cb880","Type":"ContainerDied","Data":"49839c92538026b0ce71e91aa4f51ff49f3c38cf69e1ed7e23c72e03fb70afe9"}
Jul 09 12:36:17.433785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.433415    2574 scope.go:117] "RemoveContainer" containerID="09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63"
Jul 09 12:36:17.444990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.444965    2574 scope.go:117] "RemoveContainer" containerID="09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63"
Jul 09 12:36:17.445283 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:36:17.445261    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63\": container with ID starting with 09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63 not found: ID does not exist" containerID="09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63"
Jul 09 12:36:17.445362 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.445288    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63"} err="failed to get container status \"09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63\": rpc error: code = NotFound desc = could not find container \"09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63\": container with ID starting with 09334e2cbf86fccb71bf6a6e17858126f223f68e233482b7bb3dcab6aa07ad63 not found: ID does not exist"
Jul 09 12:36:17.482456 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.482405    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bb55c7cb8-6xrb5"]
Jul 09 12:36:17.491447 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:17.491412    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bb55c7cb8-6xrb5"]
Jul 09 12:36:18.971136 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:36:18.971102    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4618d08b-a24e-4428-a98c-2179142cb880" path="/var/lib/kubelet/pods/4618d08b-a24e-4428-a98c-2179142cb880/volumes"
Jul 09 12:39:02.919855 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:39:02.919822    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:39:02.920328 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:39:02.920205    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:44:02.948865 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:44:02.948834    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:44:02.951273 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:44:02.951247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:49:02.982537 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:02.982506    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:49:02.987852 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:02.987829    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:49:59.135854 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.135819    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"]
Jul 09 12:49:59.136268 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.136212    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4618d08b-a24e-4428-a98c-2179142cb880" containerName="authorino"
Jul 09 12:49:59.136268 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.136223    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4618d08b-a24e-4428-a98c-2179142cb880" containerName="authorino"
Jul 09 12:49:59.136347 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.136315    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4618d08b-a24e-4428-a98c-2179142cb880" containerName="authorino"
Jul 09 12:49:59.138888 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.138867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.141918 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.141895    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-48e9e3cb-openshift-default-dockercfg-l9qjr\""
Jul 09 12:49:59.172226 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.172190    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"]
Jul 09 12:49:59.289628 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-data\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.289851 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-envoy\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.289851 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-token\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.289851 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-credential-socket\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.289851 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289790    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d23929-3fd1-4189-be6d-aa78f249e95b-istiod-ca-cert\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.290026 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-socket\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.290026 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289911    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-certs\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.290026 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.289927    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxq7z\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-kube-api-access-nxq7z\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.290026 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.290001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-podinfo\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391237 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-data\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391237 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-envoy\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391461 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-token\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391461 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391373    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-credential-socket\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391461 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d23929-3fd1-4189-be6d-aa78f249e95b-istiod-ca-cert\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391461 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-socket\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391483    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-certs\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxq7z\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-kube-api-access-nxq7z\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-podinfo\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391724    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-data\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.391963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.391937    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-socket\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.392042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.392020    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-certs\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.392100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.392084    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-credential-socket\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.392257 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.392238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d23929-3fd1-4189-be6d-aa78f249e95b-istiod-ca-cert\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.393556 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.393538    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-envoy\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.393952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.393936    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-podinfo\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.411140 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.411116    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-token\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.411246 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.411219    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxq7z\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-kube-api-access-nxq7z\") pod \"e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") " pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.449948 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.449911    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:49:59.607816 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.607790    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"]
Jul 09 12:49:59.610141 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:49:59.610113    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d23929_3fd1_4189_be6d_aa78f249e95b.slice/crio-4ab2f0e31332ba83e6585ccb06647263cd1e4cb9308d9b839982f0862d6d6edc WatchSource:0}: Error finding container 4ab2f0e31332ba83e6585ccb06647263cd1e4cb9308d9b839982f0862d6d6edc: Status 404 returned error can't find the container with id 4ab2f0e31332ba83e6585ccb06647263cd1e4cb9308d9b839982f0862d6d6edc
Jul 09 12:49:59.612237 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.612208    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:49:59.612331 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.612280    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:49:59.612331 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.612323    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:49:59.649581 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:49:59.649548    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw" event={"ID":"28d23929-3fd1-4189-be6d-aa78f249e95b","Type":"ContainerStarted","Data":"4ab2f0e31332ba83e6585ccb06647263cd1e4cb9308d9b839982f0862d6d6edc"}
Jul 09 12:50:00.592126 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.592089    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"]
Jul 09 12:50:00.595766 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.595746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.600182 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.600159    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-48e9e3cb-serving-cert\""
Jul 09 12:50:00.637529 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.637490    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"]
Jul 09 12:50:00.656180 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.656146    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw" event={"ID":"28d23929-3fd1-4189-be6d-aa78f249e95b","Type":"ContainerStarted","Data":"4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151"}
Jul 09 12:50:00.705588 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.705507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gmc4\" (UniqueName: \"kubernetes.io/projected/87fb9ba9-2a00-4e78-b6b4-69edf019c023-kube-api-access-2gmc4\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.705940 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.705918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87fb9ba9-2a00-4e78-b6b4-69edf019c023-maas-api-tls\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.706106 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.706091    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87fb9ba9-2a00-4e78-b6b4-69edf019c023-openshift-service-ca\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.725585 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.725527    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw" podStartSLOduration=1.725511467 podStartE2EDuration="1.725511467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:00.725345151 +0000 UTC m=+1558.319969233" watchObservedRunningTime="2026-07-09 12:50:00.725511467 +0000 UTC m=+1558.320135509"
Jul 09 12:50:00.807074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.807025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87fb9ba9-2a00-4e78-b6b4-69edf019c023-maas-api-tls\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.807074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.807073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87fb9ba9-2a00-4e78-b6b4-69edf019c023-openshift-service-ca\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.807351 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.807178    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gmc4\" (UniqueName: \"kubernetes.io/projected/87fb9ba9-2a00-4e78-b6b4-69edf019c023-kube-api-access-2gmc4\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.807894 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.807869    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87fb9ba9-2a00-4e78-b6b4-69edf019c023-openshift-service-ca\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.809562 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.809537    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87fb9ba9-2a00-4e78-b6b4-69edf019c023-maas-api-tls\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.817493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.817457    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gmc4\" (UniqueName: \"kubernetes.io/projected/87fb9ba9-2a00-4e78-b6b4-69edf019c023-kube-api-access-2gmc4\") pod \"maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:00.907182 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:00.907088    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:01.059047 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.059020    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"]
Jul 09 12:50:01.060134 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:50:01.060106    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fb9ba9_2a00_4e78_b6b4_69edf019c023.slice/crio-1c0ed3b555f4c4c6f74043290b2e2f7d4e3504e931a2e7296463d924004451bd WatchSource:0}: Error finding container 1c0ed3b555f4c4c6f74043290b2e2f7d4e3504e931a2e7296463d924004451bd: Status 404 returned error can't find the container with id 1c0ed3b555f4c4c6f74043290b2e2f7d4e3504e931a2e7296463d924004451bd
Jul 09 12:50:01.061563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.061546    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:50:01.450630 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.450596    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:50:01.455263 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.455236    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:50:01.669173 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.669132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k" event={"ID":"87fb9ba9-2a00-4e78-b6b4-69edf019c023","Type":"ContainerStarted","Data":"1c0ed3b555f4c4c6f74043290b2e2f7d4e3504e931a2e7296463d924004451bd"}
Jul 09 12:50:01.669630 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.669493    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:50:01.670406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:01.670384    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:50:02.675971 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:02.675938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k" event={"ID":"87fb9ba9-2a00-4e78-b6b4-69edf019c023","Type":"ContainerStarted","Data":"114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb"}
Jul 09 12:50:02.676451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:02.676037    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:02.698463 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:02.698408    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k" podStartSLOduration=1.824563865 podStartE2EDuration="2.698389329s" totalImagesPullingTime="873.825464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:02.695128758 +0000 UTC m=+1560.289752802" watchObservedRunningTime="2026-07-09 12:50:02.698389329 +0000 UTC m=+1560.293013371"
Jul 09 12:50:05.700128 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:05.700096    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"]
Jul 09 12:50:05.700569 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:05.700331    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw" podUID="28d23929-3fd1-4189-be6d-aa78f249e95b" containerName="istio-proxy" containerID="cri-o://4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151" gracePeriod=30
Jul 09 12:50:06.014126 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.014036    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"]
Jul 09 12:50:06.014290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.014236    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k" podUID="87fb9ba9-2a00-4e78-b6b4-69edf019c023" containerName="maas-api" containerID="cri-o://114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb" gracePeriod=30
Jul 09 12:50:06.020445 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.020417    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:06.269438 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.269368    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:06.463213 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.463175    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87fb9ba9-2a00-4e78-b6b4-69edf019c023-maas-api-tls\") pod \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") "
Jul 09 12:50:06.463383 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.463296    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87fb9ba9-2a00-4e78-b6b4-69edf019c023-openshift-service-ca\") pod \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") "
Jul 09 12:50:06.463383 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.463326    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gmc4\" (UniqueName: \"kubernetes.io/projected/87fb9ba9-2a00-4e78-b6b4-69edf019c023-kube-api-access-2gmc4\") pod \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\" (UID: \"87fb9ba9-2a00-4e78-b6b4-69edf019c023\") "
Jul 09 12:50:06.463724 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.463653    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87fb9ba9-2a00-4e78-b6b4-69edf019c023-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87fb9ba9-2a00-4e78-b6b4-69edf019c023" (UID: "87fb9ba9-2a00-4e78-b6b4-69edf019c023"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:50:06.465579 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.465546    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87fb9ba9-2a00-4e78-b6b4-69edf019c023-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87fb9ba9-2a00-4e78-b6b4-69edf019c023" (UID: "87fb9ba9-2a00-4e78-b6b4-69edf019c023"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:50:06.465728 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.465557    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87fb9ba9-2a00-4e78-b6b4-69edf019c023-kube-api-access-2gmc4" (OuterVolumeSpecName: "kube-api-access-2gmc4") pod "87fb9ba9-2a00-4e78-b6b4-69edf019c023" (UID: "87fb9ba9-2a00-4e78-b6b4-69edf019c023"). InnerVolumeSpecName "kube-api-access-2gmc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:06.564560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.564463    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gmc4\" (UniqueName: \"kubernetes.io/projected/87fb9ba9-2a00-4e78-b6b4-69edf019c023-kube-api-access-2gmc4\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:06.564560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.564499    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87fb9ba9-2a00-4e78-b6b4-69edf019c023-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:06.564560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.564509    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87fb9ba9-2a00-4e78-b6b4-69edf019c023-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:06.700605 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.700567    2574 generic.go:356] "Generic (PLEG): container finished" podID="87fb9ba9-2a00-4e78-b6b4-69edf019c023" containerID="114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb" exitCode=0
Jul 09 12:50:06.700605 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.700609    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"
Jul 09 12:50:06.701061 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.700650    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k" event={"ID":"87fb9ba9-2a00-4e78-b6b4-69edf019c023","Type":"ContainerDied","Data":"114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb"}
Jul 09 12:50:06.701061 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.700723    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k" event={"ID":"87fb9ba9-2a00-4e78-b6b4-69edf019c023","Type":"ContainerDied","Data":"1c0ed3b555f4c4c6f74043290b2e2f7d4e3504e931a2e7296463d924004451bd"}
Jul 09 12:50:06.701061 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.700748    2574 scope.go:117] "RemoveContainer" containerID="114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb"
Jul 09 12:50:06.710100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.710084    2574 scope.go:117] "RemoveContainer" containerID="114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb"
Jul 09 12:50:06.710363 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:06.710346    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb\": container with ID starting with 114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb not found: ID does not exist" containerID="114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb"
Jul 09 12:50:06.710406 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.710373    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb"} err="failed to get container status \"114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb\": rpc error: code = NotFound desc = could not find container \"114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb\": container with ID starting with 114c6a85a67a609f255558ee051c229e7df1089a7483741f77cd26a5784e8ccb not found: ID does not exist"
Jul 09 12:50:06.729137 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.729114    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"]
Jul 09 12:50:06.730889 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.730867    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48e9e3cb-5cc84d5fdb-87x5k"]
Jul 09 12:50:06.970668 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:06.970626    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87fb9ba9-2a00-4e78-b6b4-69edf019c023" path="/var/lib/kubelet/pods/87fb9ba9-2a00-4e78-b6b4-69edf019c023/volumes"
Jul 09 12:50:10.971089 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:10.971007    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:50:11.000949 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.000920    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-data\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001116 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.000961    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-podinfo\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001116 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.000980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-token\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001116 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001006    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-envoy\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001116 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001024    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxq7z\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-kube-api-access-nxq7z\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001320 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001171    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-socket\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001320 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001242    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-certs\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001320 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001249    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-data" (OuterVolumeSpecName: "istio-data") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:11.001320 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001308    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-credential-socket\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001364    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d23929-3fd1-4189-be6d-aa78f249e95b-istiod-ca-cert\") pod \"28d23929-3fd1-4189-be6d-aa78f249e95b\" (UID: \"28d23929-3fd1-4189-be6d-aa78f249e95b\") "
Jul 09 12:50:11.001521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001453    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:11.001614 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001534    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:11.001705 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001632    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:11.001847 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001825    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.001906 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001849    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.001906 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001854    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d23929-3fd1-4189-be6d-aa78f249e95b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:50:11.001906 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001864    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.001906 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.001895    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.003380 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.003345    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:50:11.003486 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.003414    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-token" (OuterVolumeSpecName: "istio-token") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:11.003486 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.003438    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:11.003486 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.003442    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-kube-api-access-nxq7z" (OuterVolumeSpecName: "kube-api-access-nxq7z") pod "28d23929-3fd1-4189-be6d-aa78f249e95b" (UID: "28d23929-3fd1-4189-be6d-aa78f249e95b"). InnerVolumeSpecName "kube-api-access-nxq7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:11.102371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.102330    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d23929-3fd1-4189-be6d-aa78f249e95b-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.102371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.102363    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.102371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.102372    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.102371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.102381    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d23929-3fd1-4189-be6d-aa78f249e95b-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.102371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.102391    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxq7z\" (UniqueName: \"kubernetes.io/projected/28d23929-3fd1-4189-be6d-aa78f249e95b-kube-api-access-nxq7z\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:11.737121 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.737083    2574 generic.go:356] "Generic (PLEG): container finished" podID="28d23929-3fd1-4189-be6d-aa78f249e95b" containerID="4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151" exitCode=0
Jul 09 12:50:11.737326 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.737131    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"
Jul 09 12:50:11.737326 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.737132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw" event={"ID":"28d23929-3fd1-4189-be6d-aa78f249e95b","Type":"ContainerDied","Data":"4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151"}
Jul 09 12:50:11.737326 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.737245    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw" event={"ID":"28d23929-3fd1-4189-be6d-aa78f249e95b","Type":"ContainerDied","Data":"4ab2f0e31332ba83e6585ccb06647263cd1e4cb9308d9b839982f0862d6d6edc"}
Jul 09 12:50:11.737326 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.737276    2574 scope.go:117] "RemoveContainer" containerID="4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151"
Jul 09 12:50:11.746883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.746864    2574 scope.go:117] "RemoveContainer" containerID="4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151"
Jul 09 12:50:11.747160 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:11.747141    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151\": container with ID starting with 4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151 not found: ID does not exist" containerID="4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151"
Jul 09 12:50:11.747210 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.747169    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151"} err="failed to get container status \"4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151\": rpc error: code = NotFound desc = could not find container \"4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151\": container with ID starting with 4ae5be63cb25df4aebbbdecfc3f68368e9d00bdd41fc90a8e261eaa4231ac151 not found: ID does not exist"
Jul 09 12:50:11.779839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.779809    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"]
Jul 09 12:50:11.792282 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:11.792254    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-48e9e3cb-openshift-default-9dcb977c-8xnkw"]
Jul 09 12:50:12.971998 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:12.971962    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d23929-3fd1-4189-be6d-aa78f249e95b" path="/var/lib/kubelet/pods/28d23929-3fd1-4189-be6d-aa78f249e95b/volumes"
Jul 09 12:50:13.756886 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.756847    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"]
Jul 09 12:50:13.757257 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.757245    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d23929-3fd1-4189-be6d-aa78f249e95b" containerName="istio-proxy"
Jul 09 12:50:13.757295 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.757259    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d23929-3fd1-4189-be6d-aa78f249e95b" containerName="istio-proxy"
Jul 09 12:50:13.757295 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.757268    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87fb9ba9-2a00-4e78-b6b4-69edf019c023" containerName="maas-api"
Jul 09 12:50:13.757295 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.757273    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="87fb9ba9-2a00-4e78-b6b4-69edf019c023" containerName="maas-api"
Jul 09 12:50:13.757385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.757354    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d23929-3fd1-4189-be6d-aa78f249e95b" containerName="istio-proxy"
Jul 09 12:50:13.757385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.757363    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="87fb9ba9-2a00-4e78-b6b4-69edf019c023" containerName="maas-api"
Jul 09 12:50:13.761801 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.761775    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.764407 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.764379    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bc6ce47b-openshift-default-dockercfg-jwf8k\""
Jul 09 12:50:13.777972 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.777942    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"]
Jul 09 12:50:13.823775 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-socket\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.823952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823785    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de69454-2503-4364-a756-19972c4eb1bb-istio-podinfo\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.823952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823815    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-istio-token\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.823952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-certs\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.823952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823882    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-envoy\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.823952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de69454-2503-4364-a756-19972c4eb1bb-istiod-ca-cert\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.823952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4gqf\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-kube-api-access-n4gqf\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.824167 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-credential-socket\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.824167 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.823977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-data\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925090 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4gqf\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-kube-api-access-n4gqf\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925090 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-credential-socket\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925133    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-data\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-socket\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de69454-2503-4364-a756-19972c4eb1bb-istio-podinfo\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-istio-token\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925278    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-certs\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-envoy\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925662 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de69454-2503-4364-a756-19972c4eb1bb-istiod-ca-cert\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925662 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925575    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-credential-socket\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925663    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-data\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925729    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-socket\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.925820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.925749    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-certs\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.926142 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.926123    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de69454-2503-4364-a756-19972c4eb1bb-istiod-ca-cert\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.927641 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.927611    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-envoy\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.927804 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.927784    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de69454-2503-4364-a756-19972c4eb1bb-istio-podinfo\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.942585 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.942555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4gqf\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-kube-api-access-n4gqf\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:13.944506 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:13.944481    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-istio-token\") pod \"e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") " pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:14.072557 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.072452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:14.223302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.223278    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"]
Jul 09 12:50:14.224530 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:50:14.224506    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de69454_2503_4364_a756_19972c4eb1bb.slice/crio-6560dff42822cba19d44b0670e5f4d87a4e089e4e34ddfdab19af33212adce9e WatchSource:0}: Error finding container 6560dff42822cba19d44b0670e5f4d87a4e089e4e34ddfdab19af33212adce9e: Status 404 returned error can't find the container with id 6560dff42822cba19d44b0670e5f4d87a4e089e4e34ddfdab19af33212adce9e
Jul 09 12:50:14.226492 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.226458    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:50:14.226590 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.226537    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:50:14.226590 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.226578    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:50:14.754471 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.754432    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" event={"ID":"6de69454-2503-4364-a756-19972c4eb1bb","Type":"ContainerStarted","Data":"94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350"}
Jul 09 12:50:14.754658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.754532    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" event={"ID":"6de69454-2503-4364-a756-19972c4eb1bb","Type":"ContainerStarted","Data":"6560dff42822cba19d44b0670e5f4d87a4e089e4e34ddfdab19af33212adce9e"}
Jul 09 12:50:14.784092 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:14.784034    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" podStartSLOduration=1.784018627 podStartE2EDuration="1.784018627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:14.78359047 +0000 UTC m=+1572.378214523" watchObservedRunningTime="2026-07-09 12:50:14.784018627 +0000 UTC m=+1572.378642670"
Jul 09 12:50:15.008236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.008148    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"]
Jul 09 12:50:15.011841 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.011819    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.014493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.014470    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bc6ce47b-serving-cert\""
Jul 09 12:50:15.023106 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.023078    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"]
Jul 09 12:50:15.037707 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.037651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.037952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.037719    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-openshift-service-ca\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.037952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.037819    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfrsv\" (UniqueName: \"kubernetes.io/projected/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-kube-api-access-dfrsv\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.073243 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.073207    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:15.078080 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.078051    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:15.138434 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.138391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfrsv\" (UniqueName: \"kubernetes.io/projected/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-kube-api-access-dfrsv\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.138617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.138506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.138617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.138538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-openshift-service-ca\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.138725 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:15.138674    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-serving-cert: secret "maas-api-e2e-ait-bc6ce47b-serving-cert" not found
Jul 09 12:50:15.138780 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:15.138768    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls podName:cf97f4a2-c3f1-4463-8f5c-be3ac10736aa nodeName:}" failed. No retries permitted until 2026-07-09 12:50:15.638746594 +0000 UTC m=+1573.233370618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls") pod "maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" (UID: "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa") : secret "maas-api-e2e-ait-bc6ce47b-serving-cert" not found
Jul 09 12:50:15.139314 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.139294    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-openshift-service-ca\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.150340 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.150311    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfrsv\" (UniqueName: \"kubernetes.io/projected/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-kube-api-access-dfrsv\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.643892 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.643855    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.646310 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.646279    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls\") pod \"maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:15.760101 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.760065    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:15.761069 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.761046    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:15.924944 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:15.924913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:16.079462 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:16.079343    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"]
Jul 09 12:50:16.081706 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:50:16.081665    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf97f4a2_c3f1_4463_8f5c_be3ac10736aa.slice/crio-cfb2adbc8b78b35878fe2f119ca56190246a978f5808ae392ec0e50c8f6ca1d8 WatchSource:0}: Error finding container cfb2adbc8b78b35878fe2f119ca56190246a978f5808ae392ec0e50c8f6ca1d8: Status 404 returned error can't find the container with id cfb2adbc8b78b35878fe2f119ca56190246a978f5808ae392ec0e50c8f6ca1d8
Jul 09 12:50:16.767910 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:16.767862    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" event={"ID":"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa","Type":"ContainerStarted","Data":"b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788"}
Jul 09 12:50:16.768085 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:16.767937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" event={"ID":"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa","Type":"ContainerStarted","Data":"cfb2adbc8b78b35878fe2f119ca56190246a978f5808ae392ec0e50c8f6ca1d8"}
Jul 09 12:50:16.768169 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:16.768131    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:16.796533 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:16.796473    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" podStartSLOduration=2.2960284189999998 podStartE2EDuration="2.796455581s" totalImagesPullingTime="500.427162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:16.793976743 +0000 UTC m=+1574.388600785" watchObservedRunningTime="2026-07-09 12:50:16.796455581 +0000 UTC m=+1574.391079623"
Jul 09 12:50:20.685869 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.685838    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"]
Jul 09 12:50:20.686279 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.686021    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" podUID="cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" containerName="maas-api" containerID="cri-o://b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788" gracePeriod=30
Jul 09 12:50:20.956120 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.956090    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:20.992770 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.992739    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfrsv\" (UniqueName: \"kubernetes.io/projected/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-kube-api-access-dfrsv\") pod \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") "
Jul 09 12:50:20.992939 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.992815    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls\") pod \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") "
Jul 09 12:50:20.992939 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.992891    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-openshift-service-ca\") pod \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\" (UID: \"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa\") "
Jul 09 12:50:20.993658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.993275    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" (UID: "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:50:20.993658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.993425    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:20.995213 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.995185    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" (UID: "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:50:20.995316 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:20.995298    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-kube-api-access-dfrsv" (OuterVolumeSpecName: "kube-api-access-dfrsv") pod "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" (UID: "cf97f4a2-c3f1-4463-8f5c-be3ac10736aa"). InnerVolumeSpecName "kube-api-access-dfrsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:21.094891 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.094849    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfrsv\" (UniqueName: \"kubernetes.io/projected/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-kube-api-access-dfrsv\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:21.094891 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.094884    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:21.798738 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.798705    2574 generic.go:356] "Generic (PLEG): container finished" podID="cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" containerID="b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788" exitCode=0
Jul 09 12:50:21.799168 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.798777    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" event={"ID":"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa","Type":"ContainerDied","Data":"b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788"}
Jul 09 12:50:21.799168 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.798818    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"
Jul 09 12:50:21.799168 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.798830    2574 scope.go:117] "RemoveContainer" containerID="b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788"
Jul 09 12:50:21.799168 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.798821    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v" event={"ID":"cf97f4a2-c3f1-4463-8f5c-be3ac10736aa","Type":"ContainerDied","Data":"cfb2adbc8b78b35878fe2f119ca56190246a978f5808ae392ec0e50c8f6ca1d8"}
Jul 09 12:50:21.807903 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.807879    2574 scope.go:117] "RemoveContainer" containerID="b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788"
Jul 09 12:50:21.808155 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:21.808135    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788\": container with ID starting with b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788 not found: ID does not exist" containerID="b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788"
Jul 09 12:50:21.808239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.808167    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788"} err="failed to get container status \"b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788\": rpc error: code = NotFound desc = could not find container \"b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788\": container with ID starting with b9d3cfb3313f2dd9aa7a0dbc66de29478139f649528d0eb92cc8591a754ee788 not found: ID does not exist"
Jul 09 12:50:21.823317 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.823290    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"]
Jul 09 12:50:21.827536 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:21.827513    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc6ce47b-b56c4cdbc-z4l7v"]
Jul 09 12:50:22.971336 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:22.971303    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" path="/var/lib/kubelet/pods/cf97f4a2-c3f1-4463-8f5c-be3ac10736aa/volumes"
Jul 09 12:50:26.350666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:26.350629    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"]
Jul 09 12:50:26.351106 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:26.350858    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" podUID="6de69454-2503-4364-a756-19972c4eb1bb" containerName="istio-proxy" containerID="cri-o://94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350" gracePeriod=30
Jul 09 12:50:30.762414 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:30.762364    2574 patch_prober.go:28] interesting pod/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 12:50:30.762932 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:30.762432    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" podUID="6de69454-2503-4364-a756-19972c4eb1bb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:50:31.618128 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.618066    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:31.695893 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.695852    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-credential-socket\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.695893 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.695901    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-istio-token\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.695933    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de69454-2503-4364-a756-19972c4eb1bb-istiod-ca-cert\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.695954    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-certs\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.695985    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4gqf\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-kube-api-access-n4gqf\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696011    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-data\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696046    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de69454-2503-4364-a756-19972c4eb1bb-istio-podinfo\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696076    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-socket\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696150 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696149    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-envoy\") pod \"6de69454-2503-4364-a756-19972c4eb1bb\" (UID: \"6de69454-2503-4364-a756-19972c4eb1bb\") "
Jul 09 12:50:31.696540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696143    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:31.696540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696447    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:31.696540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696460    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de69454-2503-4364-a756-19972c4eb1bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:50:31.696540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696286    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:31.696862 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.696832    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:31.698651 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.698542    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-kube-api-access-n4gqf" (OuterVolumeSpecName: "kube-api-access-n4gqf") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "kube-api-access-n4gqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:31.698887 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.698858    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6de69454-2503-4364-a756-19972c4eb1bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:50:31.698887 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.698862    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:31.699049 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.698865    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "6de69454-2503-4364-a756-19972c4eb1bb" (UID: "6de69454-2503-4364-a756-19972c4eb1bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:31.797194 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797155    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de69454-2503-4364-a756-19972c4eb1bb-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797194 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797187    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797194 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797197    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4gqf\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-kube-api-access-n4gqf\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797206    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797215    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de69454-2503-4364-a756-19972c4eb1bb-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797222    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797232    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797239    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de69454-2503-4364-a756-19972c4eb1bb-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.797627 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.797247    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de69454-2503-4364-a756-19972c4eb1bb-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:31.853824 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.853783    2574 generic.go:356] "Generic (PLEG): container finished" podID="6de69454-2503-4364-a756-19972c4eb1bb" containerID="94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350" exitCode=0
Jul 09 12:50:31.853824 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.853829    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"
Jul 09 12:50:31.854039 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.853871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" event={"ID":"6de69454-2503-4364-a756-19972c4eb1bb","Type":"ContainerDied","Data":"94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350"}
Jul 09 12:50:31.854039 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.853954    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq" event={"ID":"6de69454-2503-4364-a756-19972c4eb1bb","Type":"ContainerDied","Data":"6560dff42822cba19d44b0670e5f4d87a4e089e4e34ddfdab19af33212adce9e"}
Jul 09 12:50:31.854039 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.854008    2574 scope.go:117] "RemoveContainer" containerID="94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350"
Jul 09 12:50:31.863652 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.863634    2574 scope.go:117] "RemoveContainer" containerID="94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350"
Jul 09 12:50:31.863960 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:31.863941    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350\": container with ID starting with 94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350 not found: ID does not exist" containerID="94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350"
Jul 09 12:50:31.864034 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.863968    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350"} err="failed to get container status \"94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350\": rpc error: code = NotFound desc = could not find container \"94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350\": container with ID starting with 94760f801bf3b7364d637c7d440aa37e676403ddf9bd6dedaeafa29866f0f350 not found: ID does not exist"
Jul 09 12:50:31.895487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.895411    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"]
Jul 09 12:50:31.909775 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:31.909742    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bc6ce47b-openshift-default-5b97569fb7-6p7lq"]
Jul 09 12:50:32.972074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:32.972043    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6de69454-2503-4364-a756-19972c4eb1bb" path="/var/lib/kubelet/pods/6de69454-2503-4364-a756-19972c4eb1bb/volumes"
Jul 09 12:50:34.647271 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647242    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"]
Jul 09 12:50:34.647743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647714    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6de69454-2503-4364-a756-19972c4eb1bb" containerName="istio-proxy"
Jul 09 12:50:34.647743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647730    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de69454-2503-4364-a756-19972c4eb1bb" containerName="istio-proxy"
Jul 09 12:50:34.647743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647745    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" containerName="maas-api"
Jul 09 12:50:34.647931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647751    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" containerName="maas-api"
Jul 09 12:50:34.647931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647827    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf97f4a2-c3f1-4463-8f5c-be3ac10736aa" containerName="maas-api"
Jul 09 12:50:34.647931 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.647841    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6de69454-2503-4364-a756-19972c4eb1bb" containerName="istio-proxy"
Jul 09 12:50:34.650935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.650913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.654202 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.654176    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3b05b329-openshift-default-dockercfg-jkxck\""
Jul 09 12:50:34.666710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.666672    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"]
Jul 09 12:50:34.726008 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.725967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4179db16-538e-4aaf-8626-552fcbec472a-istio-podinfo\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726183 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-istio-token\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726183 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-credential-socket\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726265 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-envoy\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726265 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4179db16-538e-4aaf-8626-552fcbec472a-istiod-ca-cert\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726265 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-data\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726265 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-certs\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726391 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2w2k\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-kube-api-access-w2w2k\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.726391 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.726318    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-socket\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827350 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-certs\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827350 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2w2k\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-kube-api-access-w2w2k\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-socket\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827435    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4179db16-538e-4aaf-8626-552fcbec472a-istio-podinfo\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-istio-token\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827598 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827587    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-credential-socket\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827873 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827613    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-envoy\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827873 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827648    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4179db16-538e-4aaf-8626-552fcbec472a-istiod-ca-cert\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827873 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827735    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-data\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.827873 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827837    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-certs\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.828059 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.827877    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-socket\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.828099 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.828051    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-credential-socket\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.828099 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.828064    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-data\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.828319 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.828300    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4179db16-538e-4aaf-8626-552fcbec472a-istiod-ca-cert\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.829918 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.829896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-envoy\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.830097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.830079    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4179db16-538e-4aaf-8626-552fcbec472a-istio-podinfo\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.838367 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.838341    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-istio-token\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.838609 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.838587    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2w2k\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-kube-api-access-w2w2k\") pod \"e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") " pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:34.961252 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:34.961213    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:35.142536 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.142497    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"]
Jul 09 12:50:35.149865 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:50:35.146596    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4179db16_538e_4aaf_8626_552fcbec472a.slice/crio-541d8c4651b432d99c773f91714765dbe171131dff5ddba86e8953363a6084d3 WatchSource:0}: Error finding container 541d8c4651b432d99c773f91714765dbe171131dff5ddba86e8953363a6084d3: Status 404 returned error can't find the container with id 541d8c4651b432d99c773f91714765dbe171131dff5ddba86e8953363a6084d3
Jul 09 12:50:35.151923 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.151881    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:50:35.152017 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.151972    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:50:35.152056 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.152012    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:50:35.878691 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.878644    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55" event={"ID":"4179db16-538e-4aaf-8626-552fcbec472a","Type":"ContainerStarted","Data":"2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d"}
Jul 09 12:50:35.879079 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.878718    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55" event={"ID":"4179db16-538e-4aaf-8626-552fcbec472a","Type":"ContainerStarted","Data":"541d8c4651b432d99c773f91714765dbe171131dff5ddba86e8953363a6084d3"}
Jul 09 12:50:35.904194 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.904136    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55" podStartSLOduration=1.9041183739999998 podStartE2EDuration="1.904118374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:35.902759271 +0000 UTC m=+1593.497383343" watchObservedRunningTime="2026-07-09 12:50:35.904118374 +0000 UTC m=+1593.498742437"
Jul 09 12:50:35.927877 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.927831    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"]
Jul 09 12:50:35.931901 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.931882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:35.934572 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.934550    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-3b05b329-serving-cert\""
Jul 09 12:50:35.945451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.945423    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"]
Jul 09 12:50:35.961640 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:35.961612    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:36.040362 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.039780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca571a7b-ea9a-4d4d-848e-31fe9108d966-maas-api-tls\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.040362 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.039916    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rdxk\" (UniqueName: \"kubernetes.io/projected/ca571a7b-ea9a-4d4d-848e-31fe9108d966-kube-api-access-2rdxk\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.040362 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.039989    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca571a7b-ea9a-4d4d-848e-31fe9108d966-openshift-service-ca\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.141415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.141317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rdxk\" (UniqueName: \"kubernetes.io/projected/ca571a7b-ea9a-4d4d-848e-31fe9108d966-kube-api-access-2rdxk\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.141415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.141380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca571a7b-ea9a-4d4d-848e-31fe9108d966-openshift-service-ca\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.141655 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.141452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca571a7b-ea9a-4d4d-848e-31fe9108d966-maas-api-tls\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.142088 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.142063    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca571a7b-ea9a-4d4d-848e-31fe9108d966-openshift-service-ca\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.144002 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.143978    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca571a7b-ea9a-4d4d-848e-31fe9108d966-maas-api-tls\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.150297 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.150273    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rdxk\" (UniqueName: \"kubernetes.io/projected/ca571a7b-ea9a-4d4d-848e-31fe9108d966-kube-api-access-2rdxk\") pod \"maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.244452 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.244413    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.392928 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.392705    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"]
Jul 09 12:50:36.395577 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:50:36.395549    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca571a7b_ea9a_4d4d_848e_31fe9108d966.slice/crio-18d1867c81f5369412fd4d8d8cbad84e97f7df77f954b292a371e23e1cd475e5 WatchSource:0}: Error finding container 18d1867c81f5369412fd4d8d8cbad84e97f7df77f954b292a371e23e1cd475e5: Status 404 returned error can't find the container with id 18d1867c81f5369412fd4d8d8cbad84e97f7df77f954b292a371e23e1cd475e5
Jul 09 12:50:36.885540 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.885503    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv" event={"ID":"ca571a7b-ea9a-4d4d-848e-31fe9108d966","Type":"ContainerStarted","Data":"43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803"}
Jul 09 12:50:36.886045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.885565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv" event={"ID":"ca571a7b-ea9a-4d4d-848e-31fe9108d966","Type":"ContainerStarted","Data":"18d1867c81f5369412fd4d8d8cbad84e97f7df77f954b292a371e23e1cd475e5"}
Jul 09 12:50:36.886045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.885587    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:36.913909 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.913829    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv" podStartSLOduration=1.517231123 podStartE2EDuration="1.913808054s" totalImagesPullingTime="396.576931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:36.912655742 +0000 UTC m=+1594.507279784" watchObservedRunningTime="2026-07-09 12:50:36.913808054 +0000 UTC m=+1594.508432102"
Jul 09 12:50:36.971592 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:36.971564    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:37.890667 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:37.890631    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:37.891560 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:37.891541    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:40.452948 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:40.452898    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"]
Jul 09 12:50:40.807775 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:40.807674    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"]
Jul 09 12:50:40.807965 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:40.807938    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv" podUID="ca571a7b-ea9a-4d4d-848e-31fe9108d966" containerName="maas-api" containerID="cri-o://43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803" gracePeriod=30
Jul 09 12:50:40.914174 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:40.914126    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55" podUID="4179db16-538e-4aaf-8626-552fcbec472a" containerName="istio-proxy" containerID="cri-o://2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d" gracePeriod=30
Jul 09 12:50:41.110318 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.110293    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:41.187354 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.187312    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca571a7b-ea9a-4d4d-848e-31fe9108d966-maas-api-tls\") pod \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") "
Jul 09 12:50:41.187536 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.187452    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca571a7b-ea9a-4d4d-848e-31fe9108d966-openshift-service-ca\") pod \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") "
Jul 09 12:50:41.187536 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.187506    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rdxk\" (UniqueName: \"kubernetes.io/projected/ca571a7b-ea9a-4d4d-848e-31fe9108d966-kube-api-access-2rdxk\") pod \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\" (UID: \"ca571a7b-ea9a-4d4d-848e-31fe9108d966\") "
Jul 09 12:50:41.187897 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.187871    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca571a7b-ea9a-4d4d-848e-31fe9108d966-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca571a7b-ea9a-4d4d-848e-31fe9108d966" (UID: "ca571a7b-ea9a-4d4d-848e-31fe9108d966"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:50:41.189571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.189549    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca571a7b-ea9a-4d4d-848e-31fe9108d966-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca571a7b-ea9a-4d4d-848e-31fe9108d966" (UID: "ca571a7b-ea9a-4d4d-848e-31fe9108d966"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:50:41.189571 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.189553    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca571a7b-ea9a-4d4d-848e-31fe9108d966-kube-api-access-2rdxk" (OuterVolumeSpecName: "kube-api-access-2rdxk") pod "ca571a7b-ea9a-4d4d-848e-31fe9108d966" (UID: "ca571a7b-ea9a-4d4d-848e-31fe9108d966"). InnerVolumeSpecName "kube-api-access-2rdxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:41.288370 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.288300    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rdxk\" (UniqueName: \"kubernetes.io/projected/ca571a7b-ea9a-4d4d-848e-31fe9108d966-kube-api-access-2rdxk\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:41.288370 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.288364    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca571a7b-ea9a-4d4d-848e-31fe9108d966-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:41.288370 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.288378    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca571a7b-ea9a-4d4d-848e-31fe9108d966-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:41.920011 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.919978    2574 generic.go:356] "Generic (PLEG): container finished" podID="ca571a7b-ea9a-4d4d-848e-31fe9108d966" containerID="43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803" exitCode=0
Jul 09 12:50:41.920435 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.920028    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"
Jul 09 12:50:41.920435 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.920063    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv" event={"ID":"ca571a7b-ea9a-4d4d-848e-31fe9108d966","Type":"ContainerDied","Data":"43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803"}
Jul 09 12:50:41.920435 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.920117    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv" event={"ID":"ca571a7b-ea9a-4d4d-848e-31fe9108d966","Type":"ContainerDied","Data":"18d1867c81f5369412fd4d8d8cbad84e97f7df77f954b292a371e23e1cd475e5"}
Jul 09 12:50:41.920435 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.920140    2574 scope.go:117] "RemoveContainer" containerID="43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803"
Jul 09 12:50:41.930166 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.930145    2574 scope.go:117] "RemoveContainer" containerID="43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803"
Jul 09 12:50:41.930428 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:41.930408    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803\": container with ID starting with 43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803 not found: ID does not exist" containerID="43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803"
Jul 09 12:50:41.930500 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.930436    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803"} err="failed to get container status \"43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803\": rpc error: code = NotFound desc = could not find container \"43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803\": container with ID starting with 43ab6feabf1ec7a56451ecdf0257db6b0e723709d1fa40f632100b56dbf8c803 not found: ID does not exist"
Jul 09 12:50:41.952005 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.951954    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"]
Jul 09 12:50:41.959490 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:41.959448    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3b05b329-69cd87bdf4-4cdnv"]
Jul 09 12:50:42.971067 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:42.971034    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca571a7b-ea9a-4d4d-848e-31fe9108d966" path="/var/lib/kubelet/pods/ca571a7b-ea9a-4d4d-848e-31fe9108d966/volumes"
Jul 09 12:50:46.774712 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.774624    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:46.941804 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941771    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-istio-token\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.941804 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941810    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-credential-socket\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941850    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2w2k\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-kube-api-access-w2w2k\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941871    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4179db16-538e-4aaf-8626-552fcbec472a-istiod-ca-cert\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941891    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-data\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941916    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-socket\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941946    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4179db16-538e-4aaf-8626-552fcbec472a-istio-podinfo\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.941996    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-envoy\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942036    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-certs\") pod \"4179db16-538e-4aaf-8626-552fcbec472a\" (UID: \"4179db16-538e-4aaf-8626-552fcbec472a\") "
Jul 09 12:50:46.942309 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942102    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:46.942309 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942269    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4179db16-538e-4aaf-8626-552fcbec472a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:50:46.942411 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942396    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:46.942498 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942477    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:46.942553 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942414    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4179db16-538e-4aaf-8626-552fcbec472a-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:46.942553 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.942499    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:46.943199 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.943169    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:46.944232 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.944205    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-kube-api-access-w2w2k" (OuterVolumeSpecName: "kube-api-access-w2w2k") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "kube-api-access-w2w2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:46.944350 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.944306    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4179db16-538e-4aaf-8626-552fcbec472a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:50:46.944544 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.944521    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:50:46.944639 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.944524    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4179db16-538e-4aaf-8626-552fcbec472a" (UID: "4179db16-538e-4aaf-8626-552fcbec472a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:50:46.950580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.950441    2574 generic.go:356] "Generic (PLEG): container finished" podID="4179db16-538e-4aaf-8626-552fcbec472a" containerID="2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d" exitCode=0
Jul 09 12:50:46.950580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.950486    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"
Jul 09 12:50:46.950580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.950529    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55" event={"ID":"4179db16-538e-4aaf-8626-552fcbec472a","Type":"ContainerDied","Data":"2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d"}
Jul 09 12:50:46.950753 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.950591    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55" event={"ID":"4179db16-538e-4aaf-8626-552fcbec472a","Type":"ContainerDied","Data":"541d8c4651b432d99c773f91714765dbe171131dff5ddba86e8953363a6084d3"}
Jul 09 12:50:46.950753 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.950620    2574 scope.go:117] "RemoveContainer" containerID="2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d"
Jul 09 12:50:46.964355 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.964333    2574 scope.go:117] "RemoveContainer" containerID="2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d"
Jul 09 12:50:46.964656 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:50:46.964636    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d\": container with ID starting with 2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d not found: ID does not exist" containerID="2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d"
Jul 09 12:50:46.964762 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.964668    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d"} err="failed to get container status \"2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d\": rpc error: code = NotFound desc = could not find container \"2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d\": container with ID starting with 2f48f72aa0a818e9c796627228b9879d5a648911be905656f173f32a652e8a2d not found: ID does not exist"
Jul 09 12:50:46.976833 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.976809    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"]
Jul 09 12:50:46.981111 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:46.981090    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3b05b329-openshift-default-5d55fb98c9-7kg55"]
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043273    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043303    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2w2k\" (UniqueName: \"kubernetes.io/projected/4179db16-538e-4aaf-8626-552fcbec472a-kube-api-access-w2w2k\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043312    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043323    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043331    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4179db16-538e-4aaf-8626-552fcbec472a-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043339    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:47.043357 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:47.043347    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4179db16-538e-4aaf-8626-552fcbec472a-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:50:48.970629 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:50:48.970591    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4179db16-538e-4aaf-8626-552fcbec472a" path="/var/lib/kubelet/pods/4179db16-538e-4aaf-8626-552fcbec472a/volumes"
Jul 09 12:52:30.206605 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.206514    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"]
Jul 09 12:52:30.207093 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.207079    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4179db16-538e-4aaf-8626-552fcbec472a" containerName="istio-proxy"
Jul 09 12:52:30.207137 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.207094    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4179db16-538e-4aaf-8626-552fcbec472a" containerName="istio-proxy"
Jul 09 12:52:30.207137 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.207107    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca571a7b-ea9a-4d4d-848e-31fe9108d966" containerName="maas-api"
Jul 09 12:52:30.207137 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.207116    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca571a7b-ea9a-4d4d-848e-31fe9108d966" containerName="maas-api"
Jul 09 12:52:30.207235 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.207227    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4179db16-538e-4aaf-8626-552fcbec472a" containerName="istio-proxy"
Jul 09 12:52:30.207268 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.207241    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca571a7b-ea9a-4d4d-848e-31fe9108d966" containerName="maas-api"
Jul 09 12:52:30.210547 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.210528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.213548 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.213531    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ffdcfc07-openshift-default-dockercfg-svcp5\""
Jul 09 12:52:30.236013 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.235981    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"]
Jul 09 12:52:30.289968 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.289923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-socket\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290171 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.289999    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-envoy\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290171 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290071    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-credential-socket\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290171 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290110    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-data\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290171 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-podinfo\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290334 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-token\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290334 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-certs\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290407 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290350    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istiod-ca-cert\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.290443 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.290415    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxfbx\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-kube-api-access-gxfbx\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391209 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxfbx\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-kube-api-access-gxfbx\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391209 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-socket\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391271    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-envoy\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-credential-socket\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-data\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391337    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-podinfo\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-token\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-certs\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istiod-ca-cert\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391886 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391871    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-credential-socket\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.391953 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.391932    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-socket\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.392044 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.392023    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-data\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.392275 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.392255    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-certs\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.392318 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.392297    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istiod-ca-cert\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.393778 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.393758    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-envoy\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.393893 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.393876    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-podinfo\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.400308 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.400287    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-token\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.400530 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.400513    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxfbx\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-kube-api-access-gxfbx\") pod \"e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") " pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.523727 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.523582    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:30.675241 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.675216    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"]
Jul 09 12:52:30.677392 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:52:30.677365    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cbe98a8_8345_4e8a_a137_1d0ea025ab6e.slice/crio-1aa470cfb656b85f7d178dbb9e641f65ad4a98b9c3ce837d745ecca94d696623 WatchSource:0}: Error finding container 1aa470cfb656b85f7d178dbb9e641f65ad4a98b9c3ce837d745ecca94d696623: Status 404 returned error can't find the container with id 1aa470cfb656b85f7d178dbb9e641f65ad4a98b9c3ce837d745ecca94d696623
Jul 09 12:52:30.679513 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.679480    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:52:30.679605 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.679548    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:52:30.679605 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:30.679574    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:52:31.483887 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:31.483853    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" event={"ID":"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e","Type":"ContainerStarted","Data":"3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"}
Jul 09 12:52:31.484382 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:31.483906    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" event={"ID":"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e","Type":"ContainerStarted","Data":"1aa470cfb656b85f7d178dbb9e641f65ad4a98b9c3ce837d745ecca94d696623"}
Jul 09 12:52:31.518508 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:31.518442    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podStartSLOduration=1.51841946 podStartE2EDuration="1.51841946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:52:31.514135872 +0000 UTC m=+1709.108759914" watchObservedRunningTime="2026-07-09 12:52:31.51841946 +0000 UTC m=+1709.113043503"
Jul 09 12:52:31.524039 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:31.524018    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:31.525625 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:31.525606    2574 patch_prober.go:28] interesting pod/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 09 12:52:31.525666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:31.525648    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 09 12:52:32.528491 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:32.528455    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:33.493593 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.493561    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:33.494494 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.494476    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:33.706565 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.706527    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"]
Jul 09 12:52:33.710315 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.710289    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.713336 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.713310    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ffdcfc07-serving-cert\""
Jul 09 12:52:33.724145 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.724116    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"]
Jul 09 12:52:33.822646 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.822537    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47wlf\" (UniqueName: \"kubernetes.io/projected/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-kube-api-access-47wlf\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.822844 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.822694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-maas-api-tls\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.822844 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.822765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-openshift-service-ca\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.923250 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.923215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wlf\" (UniqueName: \"kubernetes.io/projected/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-kube-api-access-47wlf\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.923464 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.923311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-maas-api-tls\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.923464 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.923353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-openshift-service-ca\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.924080 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.924055    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-openshift-service-ca\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.926020 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.925998    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-maas-api-tls\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:33.934374 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:33.934346    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47wlf\" (UniqueName: \"kubernetes.io/projected/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-kube-api-access-47wlf\") pod \"maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:34.022201 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:34.022169    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:34.169393 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:34.169372    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"]
Jul 09 12:52:34.171714 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:52:34.171642    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a95170_99bb_4d1b_9e0c_0772da6fc2d5.slice/crio-6c93692d96be40cc96ab3bc836ed069f71ae50b97089d2418efd699d1a14ef6d WatchSource:0}: Error finding container 6c93692d96be40cc96ab3bc836ed069f71ae50b97089d2418efd699d1a14ef6d: Status 404 returned error can't find the container with id 6c93692d96be40cc96ab3bc836ed069f71ae50b97089d2418efd699d1a14ef6d
Jul 09 12:52:34.499799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:34.499760    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj" event={"ID":"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5","Type":"ContainerStarted","Data":"6c93692d96be40cc96ab3bc836ed069f71ae50b97089d2418efd699d1a14ef6d"}
Jul 09 12:52:35.506825 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:35.506785    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj" event={"ID":"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5","Type":"ContainerStarted","Data":"5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c"}
Jul 09 12:52:35.534403 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:35.534352    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj" podStartSLOduration=2.147993863 podStartE2EDuration="2.534334166s" totalImagesPullingTime="386.340303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:52:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:52:35.533191843 +0000 UTC m=+1713.127815887" watchObservedRunningTime="2026-07-09 12:52:35.534334166 +0000 UTC m=+1713.128958266"
Jul 09 12:52:36.512730 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:36.512695    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:42.522762 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:42.522724    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:46.648211 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:46.648175    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"]
Jul 09 12:52:46.648641 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:46.648375    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj" podUID="f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" containerName="maas-api" containerID="cri-o://5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c" gracePeriod=30
Jul 09 12:52:46.906151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:46.906090    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:47.053373 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.053335    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47wlf\" (UniqueName: \"kubernetes.io/projected/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-kube-api-access-47wlf\") pod \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") "
Jul 09 12:52:47.053580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.053444    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-openshift-service-ca\") pod \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") "
Jul 09 12:52:47.053580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.053472    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-maas-api-tls\") pod \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\" (UID: \"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5\") "
Jul 09 12:52:47.053935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.053905    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" (UID: "f2a95170-99bb-4d1b-9e0c-0772da6fc2d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:52:47.055570 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.055545    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" (UID: "f2a95170-99bb-4d1b-9e0c-0772da6fc2d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:52:47.055722 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.055623    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-kube-api-access-47wlf" (OuterVolumeSpecName: "kube-api-access-47wlf") pod "f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" (UID: "f2a95170-99bb-4d1b-9e0c-0772da6fc2d5"). InnerVolumeSpecName "kube-api-access-47wlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:52:47.155102 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.155058    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:52:47.155102 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.155095    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:52:47.155102 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.155108    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47wlf\" (UniqueName: \"kubernetes.io/projected/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5-kube-api-access-47wlf\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:52:47.579109 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.579070    2574 generic.go:356] "Generic (PLEG): container finished" podID="f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" containerID="5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c" exitCode=0
Jul 09 12:52:47.579290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.579114    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"
Jul 09 12:52:47.579290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.579155    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj" event={"ID":"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5","Type":"ContainerDied","Data":"5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c"}
Jul 09 12:52:47.579290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.579210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj" event={"ID":"f2a95170-99bb-4d1b-9e0c-0772da6fc2d5","Type":"ContainerDied","Data":"6c93692d96be40cc96ab3bc836ed069f71ae50b97089d2418efd699d1a14ef6d"}
Jul 09 12:52:47.579290 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.579237    2574 scope.go:117] "RemoveContainer" containerID="5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c"
Jul 09 12:52:47.581035 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.581010    2574 generic.go:356] "Generic (PLEG): container finished" podID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerID="3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1" exitCode=0
Jul 09 12:52:47.581154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.581055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" event={"ID":"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e","Type":"ContainerDied","Data":"3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"}
Jul 09 12:52:47.581418 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.581405    2574 scope.go:117] "RemoveContainer" containerID="3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"
Jul 09 12:52:47.582506 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.582473    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:52:47.582586 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.582571    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:52:47.582635 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.582609    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:52:47.588692 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.588655    2574 scope.go:117] "RemoveContainer" containerID="5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c"
Jul 09 12:52:47.588973 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:52:47.588948    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c\": container with ID starting with 5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c not found: ID does not exist" containerID="5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c"
Jul 09 12:52:47.589024 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.588986    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c"} err="failed to get container status \"5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c\": rpc error: code = NotFound desc = could not find container \"5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c\": container with ID starting with 5f6059316367ebf95c669a6c66b0c0659cd647aa6cd4b31269bbbaaabcf7567c not found: ID does not exist"
Jul 09 12:52:47.645438 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.645413    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"]
Jul 09 12:52:47.651045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:47.651016    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffdcfc07-8866db4b5-795nj"]
Jul 09 12:52:48.493554 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:48.493513    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:48.595392 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:48.595353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" event={"ID":"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e","Type":"ContainerStarted","Data":"46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99"}
Jul 09 12:52:48.972105 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:48.972066    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" path="/var/lib/kubelet/pods/f2a95170-99bb-4d1b-9e0c-0772da6fc2d5/volumes"
Jul 09 12:52:49.524097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:49.524064    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:49.525206 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:49.525177    2574 patch_prober.go:28] interesting pod/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 09 12:52:49.525333 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:49.525228    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 09 12:52:50.524482 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:50.524438    2574 patch_prober.go:28] interesting pod/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 09 12:52:50.524957 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:50.524501    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 09 12:52:51.524463 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:51.524422    2574 patch_prober.go:28] interesting pod/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 09 12:52:51.524648 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:51.524485    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 09 12:52:52.524752 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:52.524703    2574 patch_prober.go:28] interesting pod/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 09 12:52:52.525215 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:52.524773    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 09 12:52:53.527603 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:53.527564    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:53.528040 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:53.527945    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:53.528652 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:53.528631    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:52:53.680285 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:53.680243    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"]
Jul 09 12:52:54.627968 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:54.627927    2574 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" secret="" err="secret \"e2e-mt-ffdcfc07-openshift-default-dockercfg-svcp5\" not found"
Jul 09 12:52:55.632800 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:52:55.632716    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy" containerID="cri-o://46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99" gracePeriod=30
Jul 09 12:53:00.904295 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.904223    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:53:00.984776 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984742    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-socket\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.984963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984820    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istiod-ca-cert\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.984963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984838    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxfbx\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-kube-api-access-gxfbx\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.984963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-podinfo\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.984963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984875    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-credential-socket\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.984963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984894    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-envoy\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.984963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984925    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-token\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.985272 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.984996    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:53:00.985272 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985027    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-data\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.985272 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985057    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-certs\") pod \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\" (UID: \"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e\") "
Jul 09 12:53:00.985272 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985191    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:53:00.985476 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985291    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:53:00.985476 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985374    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:53:00.985584 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985494    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:00.985584 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985514    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:00.985584 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985528    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:00.985584 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.985541    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:00.986082 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.986061    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:53:00.987144 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.987118    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-kube-api-access-gxfbx" (OuterVolumeSpecName: "kube-api-access-gxfbx") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "kube-api-access-gxfbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:53:00.987445 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.987424    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:53:00.987538 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.987511    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:53:00.987578 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:00.987530    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" (UID: "9cbe98a8-8345-4e8a-a137-1d0ea025ab6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:53:01.086129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.086087    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxfbx\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-kube-api-access-gxfbx\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:01.086129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.086125    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:01.086129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.086135    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:01.086435 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.086143    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:01.086435 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.086151    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:53:01.665486 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.665440    2574 generic.go:356] "Generic (PLEG): container finished" podID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerID="46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99" exitCode=0
Jul 09 12:53:01.665710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.665490    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"
Jul 09 12:53:01.665710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.665527    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" event={"ID":"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e","Type":"ContainerDied","Data":"46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99"}
Jul 09 12:53:01.665710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.665584    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7" event={"ID":"9cbe98a8-8345-4e8a-a137-1d0ea025ab6e","Type":"ContainerDied","Data":"1aa470cfb656b85f7d178dbb9e641f65ad4a98b9c3ce837d745ecca94d696623"}
Jul 09 12:53:01.665710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.665610    2574 scope.go:117] "RemoveContainer" containerID="46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99"
Jul 09 12:53:01.674995 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.674979    2574 scope.go:117] "RemoveContainer" containerID="3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"
Jul 09 12:53:01.682943 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.682923    2574 scope.go:117] "RemoveContainer" containerID="46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99"
Jul 09 12:53:01.683199 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:53:01.683179    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99\": container with ID starting with 46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99 not found: ID does not exist" containerID="46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99"
Jul 09 12:53:01.683239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.683208    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99"} err="failed to get container status \"46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99\": rpc error: code = NotFound desc = could not find container \"46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99\": container with ID starting with 46df4181e24947dc83651e0a4c49e77b6d02a89618d0473e993d33571c1c9d99 not found: ID does not exist"
Jul 09 12:53:01.683239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.683228    2574 scope.go:117] "RemoveContainer" containerID="3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"
Jul 09 12:53:01.683481 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:53:01.683466    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1\": container with ID starting with 3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1 not found: ID does not exist" containerID="3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"
Jul 09 12:53:01.683522 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.683487    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1"} err="failed to get container status \"3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1\": rpc error: code = NotFound desc = could not find container \"3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1\": container with ID starting with 3a03d5d9fa3c1b1cc7ec2641ff5c1805d49472e97bdef68f5f3ebc29d48001b1 not found: ID does not exist"
Jul 09 12:53:01.689895 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.689871    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"]
Jul 09 12:53:01.695385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:01.695362    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ffdcfc07-openshift-default-67c67477c5-n2qv7"]
Jul 09 12:53:02.971652 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:02.971609    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" path="/var/lib/kubelet/pods/9cbe98a8-8345-4e8a-a137-1d0ea025ab6e/volumes"
Jul 09 12:53:15.352453 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.352411    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"]
Jul 09 12:53:15.352997 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.352982    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy"
Jul 09 12:53:15.353042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353000    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy"
Jul 09 12:53:15.353042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353026    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy"
Jul 09 12:53:15.353042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353034    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy"
Jul 09 12:53:15.353188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353063    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" containerName="maas-api"
Jul 09 12:53:15.353188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353072    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" containerName="maas-api"
Jul 09 12:53:15.353188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353154    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy"
Jul 09 12:53:15.353188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353164    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cbe98a8-8345-4e8a-a137-1d0ea025ab6e" containerName="istio-proxy"
Jul 09 12:53:15.353188 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.353175    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a95170-99bb-4d1b-9e0c-0772da6fc2d5" containerName="maas-api"
Jul 09 12:53:15.356518 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.356499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.363182 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:53:15.363093    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.363566 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:53:15.363530    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-mt-a0a88c96-openshift-default-dockercfg-vrnr2\" is forbidden: User \"system:node:ip-10-0-140-237.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-237.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-mt-a0a88c96-openshift-default-dockercfg-vrnr2\"" type="*v1.Secret"
Jul 09 12:53:15.384954 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.384919    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"]
Jul 09 12:53:15.423377 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423340    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-envoy\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423377 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49jxh\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-kube-api-access-49jxh\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-certs\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423482    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-podinfo\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423551    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-credential-socket\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423573    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-data\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-socket\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.423756 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-token\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.424092 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.423765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istiod-ca-cert\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524640 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-certs\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524640 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-podinfo\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524697    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-credential-socket\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524722    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-data\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-socket\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-token\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524833    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istiod-ca-cert\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-envoy\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.524958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.524935    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jxh\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-kube-api-access-49jxh\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.525314 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.525088    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-certs\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.525314 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.525176    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-socket\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.525450 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.525417    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-data\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.525594 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.525574    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-credential-socket\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.525775 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.525756    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istiod-ca-cert\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.527147 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.527128    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-envoy\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.527496 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.527478    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-podinfo\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.534088 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.534062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-token\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:15.534088 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:15.534076    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49jxh\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-kube-api-access-49jxh\") pod \"e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") " pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:16.667436 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.667397    2574 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 09 12:53:16.667941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.667486    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:16.806051 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.806023    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a0a88c96-openshift-default-dockercfg-vrnr2\""
Jul 09 12:53:16.831829 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.831803    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"]
Jul 09 12:53:16.834316 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:53:16.834215    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d61f2d_262c_4ca2_932d_5ddd6a25079b.slice/crio-cf8b3995a063106d069af29c69820b98596051ef528e0ce9af97a6c38fd6735e WatchSource:0}: Error finding container cf8b3995a063106d069af29c69820b98596051ef528e0ce9af97a6c38fd6735e: Status 404 returned error can't find the container with id cf8b3995a063106d069af29c69820b98596051ef528e0ce9af97a6c38fd6735e
Jul 09 12:53:16.837047 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.837013    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:53:16.837155 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.837090    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:53:16.837155 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:16.837119    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:53:17.755421 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:17.755382    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" event={"ID":"f4d61f2d-262c-4ca2-932d-5ddd6a25079b","Type":"ContainerStarted","Data":"02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"}
Jul 09 12:53:17.755891 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:17.755443    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" event={"ID":"f4d61f2d-262c-4ca2-932d-5ddd6a25079b","Type":"ContainerStarted","Data":"cf8b3995a063106d069af29c69820b98596051ef528e0ce9af97a6c38fd6735e"}
Jul 09 12:53:17.776476 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:17.776416    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" podStartSLOduration=2.7763968549999998 podStartE2EDuration="2.776396855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:53:17.775779694 +0000 UTC m=+1755.370403739" watchObservedRunningTime="2026-07-09 12:53:17.776396855 +0000 UTC m=+1755.371020900"
Jul 09 12:53:18.668197 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:18.668147    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:18.673261 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:18.673232    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:18.762054 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:18.762020    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:18.762981 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:18.762961    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:20.018287 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.018253    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"]
Jul 09 12:53:20.021883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.021864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.024538 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.024513    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a0a88c96-serving-cert\""
Jul 09 12:53:20.038266 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.038235    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"]
Jul 09 12:53:20.066901 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.066861    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rc6k\" (UniqueName: \"kubernetes.io/projected/181b890a-8427-401e-bd02-24bb75fea24e-kube-api-access-2rc6k\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.066901 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.066910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.067169 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.066939    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/181b890a-8427-401e-bd02-24bb75fea24e-openshift-service-ca\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.167716 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.167651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rc6k\" (UniqueName: \"kubernetes.io/projected/181b890a-8427-401e-bd02-24bb75fea24e-kube-api-access-2rc6k\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.167929 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.167754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.167929 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.167805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/181b890a-8427-401e-bd02-24bb75fea24e-openshift-service-ca\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.167929 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:53:20.167891    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-serving-cert: secret "maas-api-e2e-mt-a0a88c96-serving-cert" not found
Jul 09 12:53:20.168068 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:53:20.167954    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls podName:181b890a-8427-401e-bd02-24bb75fea24e nodeName:}" failed. No retries permitted until 2026-07-09 12:53:20.667937615 +0000 UTC m=+1758.262561636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls") pod "maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" (UID: "181b890a-8427-401e-bd02-24bb75fea24e") : secret "maas-api-e2e-mt-a0a88c96-serving-cert" not found
Jul 09 12:53:20.168473 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.168454    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/181b890a-8427-401e-bd02-24bb75fea24e-openshift-service-ca\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.180567 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.180536    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rc6k\" (UniqueName: \"kubernetes.io/projected/181b890a-8427-401e-bd02-24bb75fea24e-kube-api-access-2rc6k\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.673222 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.673188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.676029 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.675991    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls\") pod \"maas-api-e2e-mt-a0a88c96-559795ff8-2v42r\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:20.932927 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:20.932821    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:21.096443 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:21.096417    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"]
Jul 09 12:53:21.098165 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:53:21.098131    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181b890a_8427_401e_bd02_24bb75fea24e.slice/crio-01b68104ec9c5d41216a055026cb1eaee759cdac0c75459caa6ff8c12ace39fd WatchSource:0}: Error finding container 01b68104ec9c5d41216a055026cb1eaee759cdac0c75459caa6ff8c12ace39fd: Status 404 returned error can't find the container with id 01b68104ec9c5d41216a055026cb1eaee759cdac0c75459caa6ff8c12ace39fd
Jul 09 12:53:21.779578 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:21.779488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" event={"ID":"181b890a-8427-401e-bd02-24bb75fea24e","Type":"ContainerStarted","Data":"63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991"}
Jul 09 12:53:21.779578 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:21.779551    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" event={"ID":"181b890a-8427-401e-bd02-24bb75fea24e","Type":"ContainerStarted","Data":"01b68104ec9c5d41216a055026cb1eaee759cdac0c75459caa6ff8c12ace39fd"}
Jul 09 12:53:21.779820 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:21.779591    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:21.808493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:21.808439    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" podStartSLOduration=2.463199625 podStartE2EDuration="2.808423172s" totalImagesPullingTime="345.223547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:53:21.806650189 +0000 UTC m=+1759.401274230" watchObservedRunningTime="2026-07-09 12:53:21.808423172 +0000 UTC m=+1759.403047215"
Jul 09 12:53:27.790650 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:27.790618    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:53:32.836371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:32.836286    2574 generic.go:356] "Generic (PLEG): container finished" podID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerID="02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d" exitCode=0
Jul 09 12:53:32.836371 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:32.836359    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" event={"ID":"f4d61f2d-262c-4ca2-932d-5ddd6a25079b","Type":"ContainerDied","Data":"02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"}
Jul 09 12:53:32.836825 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:32.836794    2574 scope.go:117] "RemoveContainer" containerID="02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"
Jul 09 12:53:32.837925 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:32.837895    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:53:32.838045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:32.838028    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:53:32.838082 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:32.838074    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:53:33.762718 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:33.762664    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:33.848467 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:33.848416    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" event={"ID":"f4d61f2d-262c-4ca2-932d-5ddd6a25079b","Type":"ContainerStarted","Data":"eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd"}
Jul 09 12:53:34.668111 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:34.668062    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:34.669409 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:34.669380    2574 patch_prober.go:28] interesting pod/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 09 12:53:34.669530 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:34.669433    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 09 12:53:35.668308 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:35.668266    2574 patch_prober.go:28] interesting pod/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 09 12:53:35.668835 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:35.668344    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 09 12:53:36.668713 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:36.668653    2574 patch_prober.go:28] interesting pod/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 09 12:53:36.669138 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:36.668729    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 09 12:53:37.673746 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:37.673709    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:37.674196 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:37.674188    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:53:37.675074 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:53:37.675051    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:54:03.017929 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:03.017884    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:54:03.025366 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:03.025338    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:54:07.770885 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:07.770850    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"]
Jul 09 12:54:07.771447 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:07.771067    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" podUID="181b890a-8427-401e-bd02-24bb75fea24e" containerName="maas-api" containerID="cri-o://63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991" gracePeriod=30
Jul 09 12:54:07.785911 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:07.785863    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" podUID="181b890a-8427-401e-bd02-24bb75fea24e" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.59:8443/health\": dial tcp 10.132.0.59:8443: connect: connection refused"
Jul 09 12:54:08.028747 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.028660    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:54:08.034047 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.034018    2574 generic.go:356] "Generic (PLEG): container finished" podID="181b890a-8427-401e-bd02-24bb75fea24e" containerID="63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991" exitCode=0
Jul 09 12:54:08.034207 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.034057    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"
Jul 09 12:54:08.034207 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.034102    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" event={"ID":"181b890a-8427-401e-bd02-24bb75fea24e","Type":"ContainerDied","Data":"63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991"}
Jul 09 12:54:08.034207 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.034162    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r" event={"ID":"181b890a-8427-401e-bd02-24bb75fea24e","Type":"ContainerDied","Data":"01b68104ec9c5d41216a055026cb1eaee759cdac0c75459caa6ff8c12ace39fd"}
Jul 09 12:54:08.034207 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.034192    2574 scope.go:117] "RemoveContainer" containerID="63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991"
Jul 09 12:54:08.043215 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.043195    2574 scope.go:117] "RemoveContainer" containerID="63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991"
Jul 09 12:54:08.043510 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:54:08.043492    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991\": container with ID starting with 63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991 not found: ID does not exist" containerID="63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991"
Jul 09 12:54:08.043561 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.043522    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991"} err="failed to get container status \"63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991\": rpc error: code = NotFound desc = could not find container \"63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991\": container with ID starting with 63122a100e5cf77d1eceab9eef958911edf64c7dae6a1f1650d88ac533723991 not found: ID does not exist"
Jul 09 12:54:08.128492 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.128460    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls\") pod \"181b890a-8427-401e-bd02-24bb75fea24e\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") "
Jul 09 12:54:08.128704 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.128514    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/181b890a-8427-401e-bd02-24bb75fea24e-openshift-service-ca\") pod \"181b890a-8427-401e-bd02-24bb75fea24e\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") "
Jul 09 12:54:08.128704 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.128568    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rc6k\" (UniqueName: \"kubernetes.io/projected/181b890a-8427-401e-bd02-24bb75fea24e-kube-api-access-2rc6k\") pod \"181b890a-8427-401e-bd02-24bb75fea24e\" (UID: \"181b890a-8427-401e-bd02-24bb75fea24e\") "
Jul 09 12:54:08.128936 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.128913    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/181b890a-8427-401e-bd02-24bb75fea24e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "181b890a-8427-401e-bd02-24bb75fea24e" (UID: "181b890a-8427-401e-bd02-24bb75fea24e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:54:08.130623 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.130592    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181b890a-8427-401e-bd02-24bb75fea24e-kube-api-access-2rc6k" (OuterVolumeSpecName: "kube-api-access-2rc6k") pod "181b890a-8427-401e-bd02-24bb75fea24e" (UID: "181b890a-8427-401e-bd02-24bb75fea24e"). InnerVolumeSpecName "kube-api-access-2rc6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:54:08.130768 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.130624    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "181b890a-8427-401e-bd02-24bb75fea24e" (UID: "181b890a-8427-401e-bd02-24bb75fea24e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:54:08.230105 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.230069    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rc6k\" (UniqueName: \"kubernetes.io/projected/181b890a-8427-401e-bd02-24bb75fea24e-kube-api-access-2rc6k\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:54:08.230105 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.230101    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/181b890a-8427-401e-bd02-24bb75fea24e-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:54:08.230105 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.230111    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/181b890a-8427-401e-bd02-24bb75fea24e-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:54:08.358670 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.358631    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"]
Jul 09 12:54:08.362341 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.362312    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0a88c96-559795ff8-2v42r"]
Jul 09 12:54:08.971883 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:54:08.971849    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="181b890a-8427-401e-bd02-24bb75fea24e" path="/var/lib/kubelet/pods/181b890a-8427-401e-bd02-24bb75fea24e/volumes"
Jul 09 12:55:03.006564 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:03.006518    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"]
Jul 09 12:55:03.007112 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:03.006807    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy" containerID="cri-o://eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd" gracePeriod=30
Jul 09 12:55:07.676491 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:07.676452    2574 patch_prober.go:28] interesting pod/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 12:55:07.676945 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:07.676526    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:55:08.269161 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.269093    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:55:08.340645 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.340606    2574 generic.go:356] "Generic (PLEG): container finished" podID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerID="eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd" exitCode=0
Jul 09 12:55:08.340645 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.340649    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"
Jul 09 12:55:08.340935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.340691    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" event={"ID":"f4d61f2d-262c-4ca2-932d-5ddd6a25079b","Type":"ContainerDied","Data":"eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd"}
Jul 09 12:55:08.340935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.340766    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s" event={"ID":"f4d61f2d-262c-4ca2-932d-5ddd6a25079b","Type":"ContainerDied","Data":"cf8b3995a063106d069af29c69820b98596051ef528e0ce9af97a6c38fd6735e"}
Jul 09 12:55:08.340935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.340800    2574 scope.go:117] "RemoveContainer" containerID="eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd"
Jul 09 12:55:08.350513 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.350488    2574 scope.go:117] "RemoveContainer" containerID="02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"
Jul 09 12:55:08.361002 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.360977    2574 scope.go:117] "RemoveContainer" containerID="eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd"
Jul 09 12:55:08.361340 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:55:08.361320    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd\": container with ID starting with eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd not found: ID does not exist" containerID="eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd"
Jul 09 12:55:08.361408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.361351    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd"} err="failed to get container status \"eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd\": rpc error: code = NotFound desc = could not find container \"eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd\": container with ID starting with eb0e338a595c291017f233bd96a8889a7193b80a8a3283d0aeb55aac332454cd not found: ID does not exist"
Jul 09 12:55:08.361408 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.361370    2574 scope.go:117] "RemoveContainer" containerID="02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"
Jul 09 12:55:08.361588 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:55:08.361571    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d\": container with ID starting with 02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d not found: ID does not exist" containerID="02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"
Jul 09 12:55:08.361628 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.361594    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d"} err="failed to get container status \"02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d\": rpc error: code = NotFound desc = could not find container \"02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d\": container with ID starting with 02b2c0f91e87e6bfbb254cc785096b7b74fe2c447ae5c00c3dec5cbfdcf4e04d not found: ID does not exist"
Jul 09 12:55:08.396515 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396480    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-socket\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.396734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396536    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-data\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.396734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396585    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49jxh\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-kube-api-access-49jxh\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.396734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396608    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-token\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.396734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396635    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-credential-socket\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.396734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396661    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-envoy\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.396734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396714    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-certs\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.397072 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396739    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-podinfo\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.397072 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396762    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istiod-ca-cert\") pod \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\" (UID: \"f4d61f2d-262c-4ca2-932d-5ddd6a25079b\") "
Jul 09 12:55:08.397072 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.396836    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:55:08.397072 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.397042    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:55:08.397072 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.397059    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.397378 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.397344    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:55:08.397620 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.397469    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:55:08.397767 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.397645    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:55:08.399085 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.399058    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:55:08.399173 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.399067    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:55:08.399173 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.399107    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:55:08.399173 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.399138    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-kube-api-access-49jxh" (OuterVolumeSpecName: "kube-api-access-49jxh") pod "f4d61f2d-262c-4ca2-932d-5ddd6a25079b" (UID: "f4d61f2d-262c-4ca2-932d-5ddd6a25079b"). InnerVolumeSpecName "kube-api-access-49jxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:55:08.498154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498109    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498148    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49jxh\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-kube-api-access-49jxh\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498154 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498160    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498169    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498177    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498185    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498193    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.498402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.498200    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4d61f2d-262c-4ca2-932d-5ddd6a25079b-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:55:08.688659 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.688620    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"]
Jul 09 12:55:08.705661 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.705626    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a0a88c96-openshift-default-59f7f8fd5f-9cf4s"]
Jul 09 12:55:08.972029 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:55:08.971992    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" path="/var/lib/kubelet/pods/f4d61f2d-262c-4ca2-932d-5ddd6a25079b/volumes"
Jul 09 12:56:00.972181 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972142    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"]
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972499    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972509    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972520    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972525    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972533    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181b890a-8427-401e-bd02-24bb75fea24e" containerName="maas-api"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972539    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="181b890a-8427-401e-bd02-24bb75fea24e" containerName="maas-api"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972606    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy"
Jul 09 12:56:00.972612 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972613    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d61f2d-262c-4ca2-932d-5ddd6a25079b" containerName="istio-proxy"
Jul 09 12:56:00.972917 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.972620    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="181b890a-8427-401e-bd02-24bb75fea24e" containerName="maas-api"
Jul 09 12:56:00.975782 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.975762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:00.980642 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.980609    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-39792d-openshift-default-dockercfg-5j54c\""
Jul 09 12:56:00.998745 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:00.998711    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"]
Jul 09 12:56:01.166762 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-certs\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.166762 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-data\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.166991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166787    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-istio-token\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.166991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166845    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2086c1-0294-4845-a967-933635bc2f60-istiod-ca-cert\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.166991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-socket\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.166991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2086c1-0294-4845-a967-933635bc2f60-istio-podinfo\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.166991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.166977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-credential-socket\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.167156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.167041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-envoy\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.167156 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.167074    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md9zw\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-kube-api-access-md9zw\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268547 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-envoy\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268547 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md9zw\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-kube-api-access-md9zw\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268547 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-certs\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268547 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268544    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-data\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-istio-token\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2086c1-0294-4845-a967-933635bc2f60-istiod-ca-cert\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268642    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-socket\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2086c1-0294-4845-a967-933635bc2f60-istio-podinfo\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.268947 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.268745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-credential-socket\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.269216 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.269009    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-data\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.269216 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.269048    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-certs\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.269330 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.269297    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-socket\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.269415 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.269390    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-credential-socket\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.269639 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.269610    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2086c1-0294-4845-a967-933635bc2f60-istiod-ca-cert\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.270963 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.270941    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-envoy\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.271425 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.271405    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2086c1-0294-4845-a967-933635bc2f60-istio-podinfo\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.282620 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.282598    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-istio-token\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.282748 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.282726    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md9zw\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-kube-api-access-md9zw\") pod \"e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") " pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.286561 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.286543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:01.437907 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.437879    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"]
Jul 09 12:56:01.439981 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:56:01.439957    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb2086c1_0294_4845_a967_933635bc2f60.slice/crio-0d2701f49dfb3c4082fd675a625f6105ba922b0d92f88f81da9051889aff19b9 WatchSource:0}: Error finding container 0d2701f49dfb3c4082fd675a625f6105ba922b0d92f88f81da9051889aff19b9: Status 404 returned error can't find the container with id 0d2701f49dfb3c4082fd675a625f6105ba922b0d92f88f81da9051889aff19b9
Jul 09 12:56:01.448697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.442673    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:56:01.448697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.442914    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:56:01.448697 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.443025    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:56:01.621912 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.621807    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" event={"ID":"eb2086c1-0294-4845-a967-933635bc2f60","Type":"ContainerStarted","Data":"d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"}
Jul 09 12:56:01.622075 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.621918    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" event={"ID":"eb2086c1-0294-4845-a967-933635bc2f60","Type":"ContainerStarted","Data":"0d2701f49dfb3c4082fd675a625f6105ba922b0d92f88f81da9051889aff19b9"}
Jul 09 12:56:01.651537 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:01.651477    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podStartSLOduration=1.6514579 podStartE2EDuration="1.6514579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:56:01.649659502 +0000 UTC m=+1919.244283559" watchObservedRunningTime="2026-07-09 12:56:01.6514579 +0000 UTC m=+1919.246081943"
Jul 09 12:56:02.287563 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:02.287525    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:02.292636 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:02.292610    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:02.627139 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:02.627039    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:02.628007 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:02.627985    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:03.767432 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.767398    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"]
Jul 09 12:56:03.772340 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.772319    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.775642 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.775614    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-39792d-serving-cert\""
Jul 09 12:56:03.784481 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.784456    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"]
Jul 09 12:56:03.789372 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.789344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jtwl\" (UniqueName: \"kubernetes.io/projected/9b587e81-0ade-4abc-9e79-295c1224e44a-kube-api-access-2jtwl\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.789499 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.789409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.789555 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.789501    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b587e81-0ade-4abc-9e79-295c1224e44a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.890501 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.890465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.890710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.890555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b587e81-0ade-4abc-9e79-295c1224e44a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.890710 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.890601    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jtwl\" (UniqueName: \"kubernetes.io/projected/9b587e81-0ade-4abc-9e79-295c1224e44a-kube-api-access-2jtwl\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.890710 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:56:03.890626    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-serving-cert: secret "maas-api-e2e-inf-a-39792d-serving-cert" not found
Jul 09 12:56:03.890892 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:56:03.890719    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls podName:9b587e81-0ade-4abc-9e79-295c1224e44a nodeName:}" failed. No retries permitted until 2026-07-09 12:56:04.390696216 +0000 UTC m=+1921.985320239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls") pod "maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" (UID: "9b587e81-0ade-4abc-9e79-295c1224e44a") : secret "maas-api-e2e-inf-a-39792d-serving-cert" not found
Jul 09 12:56:03.891392 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.891371    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b587e81-0ade-4abc-9e79-295c1224e44a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:03.909447 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:03.909413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jtwl\" (UniqueName: \"kubernetes.io/projected/9b587e81-0ade-4abc-9e79-295c1224e44a-kube-api-access-2jtwl\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:04.395372 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:04.395330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:04.397991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:04.397963    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls\") pod \"maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:04.683360 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:04.683259    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:04.835565 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:04.835534    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"]
Jul 09 12:56:04.837995 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:56:04.837960    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b587e81_0ade_4abc_9e79_295c1224e44a.slice/crio-a9ffe088ff38a37ccff6437073b6dbf02e90abb90b47c2526c67e185f05bf539 WatchSource:0}: Error finding container a9ffe088ff38a37ccff6437073b6dbf02e90abb90b47c2526c67e185f05bf539: Status 404 returned error can't find the container with id a9ffe088ff38a37ccff6437073b6dbf02e90abb90b47c2526c67e185f05bf539
Jul 09 12:56:04.839592 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:04.839568    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:56:05.650533 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:05.650428    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" event={"ID":"9b587e81-0ade-4abc-9e79-295c1224e44a","Type":"ContainerStarted","Data":"503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976"}
Jul 09 12:56:05.650533 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:05.650499    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:05.650533 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:05.650521    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" event={"ID":"9b587e81-0ade-4abc-9e79-295c1224e44a","Type":"ContainerStarted","Data":"a9ffe088ff38a37ccff6437073b6dbf02e90abb90b47c2526c67e185f05bf539"}
Jul 09 12:56:05.670854 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:05.670801    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" podStartSLOduration=2.22599727 podStartE2EDuration="2.670781198s" totalImagesPullingTime="444.783928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:56:05.670017543 +0000 UTC m=+1923.264641638" watchObservedRunningTime="2026-07-09 12:56:05.670781198 +0000 UTC m=+1923.265405241"
Jul 09 12:56:11.661506 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:11.661472    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:56:14.713933 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.713897    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"]
Jul 09 12:56:14.717977 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.717947    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.720561 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.720537    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-cb649b-serving-cert\""
Jul 09 12:56:14.755704 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.751180    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"]
Jul 09 12:56:14.783565 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.783530    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.783788 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.783572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8fsk\" (UniqueName: \"kubernetes.io/projected/3724b5e2-89c3-4706-8872-72407200ccd1-kube-api-access-c8fsk\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.783788 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.783661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3724b5e2-89c3-4706-8872-72407200ccd1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.884629 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.884591    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3724b5e2-89c3-4706-8872-72407200ccd1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.884822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.884721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.884822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.884745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8fsk\" (UniqueName: \"kubernetes.io/projected/3724b5e2-89c3-4706-8872-72407200ccd1-kube-api-access-c8fsk\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.884898 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:56:14.884818    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-serving-cert: secret "maas-api-e2e-inf-b-cb649b-serving-cert" not found
Jul 09 12:56:14.884933 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:56:14.884898    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls podName:3724b5e2-89c3-4706-8872-72407200ccd1 nodeName:}" failed. No retries permitted until 2026-07-09 12:56:15.384875264 +0000 UTC m=+1932.979499286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls") pod "maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" (UID: "3724b5e2-89c3-4706-8872-72407200ccd1") : secret "maas-api-e2e-inf-b-cb649b-serving-cert" not found
Jul 09 12:56:14.885393 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.885376    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3724b5e2-89c3-4706-8872-72407200ccd1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:14.895492 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:14.895461    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8fsk\" (UniqueName: \"kubernetes.io/projected/3724b5e2-89c3-4706-8872-72407200ccd1-kube-api-access-c8fsk\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:15.389958 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:15.389919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:15.392466 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:15.392441    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:15.630093 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:15.630055    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:15.822914 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:15.822887    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"]
Jul 09 12:56:15.824903 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:56:15.824875    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3724b5e2_89c3_4706_8872_72407200ccd1.slice/crio-df4d8cd568a142d88a52270346b46967b18ea688a8cd2204cba427f15bbedf79 WatchSource:0}: Error finding container df4d8cd568a142d88a52270346b46967b18ea688a8cd2204cba427f15bbedf79: Status 404 returned error can't find the container with id df4d8cd568a142d88a52270346b46967b18ea688a8cd2204cba427f15bbedf79
Jul 09 12:56:16.710992 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:16.710945    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" event={"ID":"3724b5e2-89c3-4706-8872-72407200ccd1","Type":"ContainerStarted","Data":"2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7"}
Jul 09 12:56:16.711180 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:16.711052    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:16.711180 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:16.711077    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" event={"ID":"3724b5e2-89c3-4706-8872-72407200ccd1","Type":"ContainerStarted","Data":"df4d8cd568a142d88a52270346b46967b18ea688a8cd2204cba427f15bbedf79"}
Jul 09 12:56:16.740494 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:16.740436    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" podStartSLOduration=2.457841057 podStartE2EDuration="2.740418797s" totalImagesPullingTime="282.57774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:56:16.738957183 +0000 UTC m=+1934.333581246" watchObservedRunningTime="2026-07-09 12:56:16.740418797 +0000 UTC m=+1934.335042841"
Jul 09 12:56:18.667896 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:18.667845    2574 patch_prober.go:28] interesting pod/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 09 12:56:18.668302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:18.667909    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 09 12:56:19.728870 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:19.728832    2574 generic.go:356] "Generic (PLEG): container finished" podID="eb2086c1-0294-4845-a967-933635bc2f60" containerID="d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e" exitCode=0
Jul 09 12:56:19.729363 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:19.728906    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" event={"ID":"eb2086c1-0294-4845-a967-933635bc2f60","Type":"ContainerDied","Data":"d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"}
Jul 09 12:56:19.729363 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:19.729317    2574 scope.go:117] "RemoveContainer" containerID="d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"
Jul 09 12:56:19.730294 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:19.730270    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:56:19.730353 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:19.730340    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:56:19.730392 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:19.730373    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:56:20.737445 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:20.737400    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" event={"ID":"eb2086c1-0294-4845-a967-933635bc2f60","Type":"ContainerStarted","Data":"4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059"}
Jul 09 12:56:21.287345 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:21.287306    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:21.287674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:21.287649    2574 patch_prober.go:28] interesting pod/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 09 12:56:21.287813 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:21.287739    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 09 12:56:22.287701 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:22.287648    2574 patch_prober.go:28] interesting pod/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 09 12:56:22.288130 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:22.287728    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 09 12:56:22.722660 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:22.722627    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:56:23.287344 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:23.287297    2574 patch_prober.go:28] interesting pod/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 09 12:56:23.287562 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:23.287364    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 09 12:56:24.287378 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:24.287345    2574 patch_prober.go:28] interesting pod/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 09 12:56:24.287788 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:24.287411    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 09 12:56:25.290946 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:25.290912    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:25.291355 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:25.291175    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:25.292061 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:25.292046    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:56:54.763590 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.763550    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"]
Jul 09 12:56:54.767078 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.767054    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.769924 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.769897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-dshm\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.770083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.769929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tls-certs\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.770083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.769978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-model-cache\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.770083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.769999    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rw2v\" (UniqueName: \"kubernetes.io/projected/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-kube-api-access-2rw2v\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.770083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.770028    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-home\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.770256 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.770139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tmp-dir\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.775081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.774780    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cb649b\"/\"openshift-service-ca.crt\""
Jul 09 12:56:54.775081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.775004    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cb649b\"/\"test-model-cb649b-kserve-self-signed-certs\""
Jul 09 12:56:54.775081 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.775065    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cb649b\"/\"kube-root-ca.crt\""
Jul 09 12:56:54.775259 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.775097    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cb649b\"/\"default-dockercfg-hdcxd\""
Jul 09 12:56:54.804733 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.804694    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"]
Jul 09 12:56:54.870520 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-dshm\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.870520 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870512    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tls-certs\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.870822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-model-cache\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.870822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870662    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rw2v\" (UniqueName: \"kubernetes.io/projected/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-kube-api-access-2rw2v\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.870822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-home\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.870822 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tmp-dir\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.871052 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.870959    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-model-cache\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.871052 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.871028    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-home\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.871168 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.871120    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tmp-dir\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.872990 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.872966    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-dshm\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.873178 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.873159    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tls-certs\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:54.883544 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:54.883514    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rw2v\" (UniqueName: \"kubernetes.io/projected/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-kube-api-access-2rw2v\") pod \"test-model-cb649b-kserve-694fcb657c-46724\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") " pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:55.077772 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:55.077630    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:56:55.242421 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:55.242390    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"]
Jul 09 12:56:55.244222 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:56:55.244187    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62f95e95_b0d1_4df8_bd54_dcdf4f80acac.slice/crio-2c04b64b643c04f778f3fd9d1579e6925fc7c1cb3a3fc851323c6b395f590c61 WatchSource:0}: Error finding container 2c04b64b643c04f778f3fd9d1579e6925fc7c1cb3a3fc851323c6b395f590c61: Status 404 returned error can't find the container with id 2c04b64b643c04f778f3fd9d1579e6925fc7c1cb3a3fc851323c6b395f590c61
Jul 09 12:56:55.936709 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:55.936641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724" event={"ID":"62f95e95-b0d1-4df8-bd54-dcdf4f80acac","Type":"ContainerStarted","Data":"1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24"}
Jul 09 12:56:55.936709 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:55.936706    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724" event={"ID":"62f95e95-b0d1-4df8-bd54-dcdf4f80acac","Type":"ContainerStarted","Data":"2c04b64b643c04f778f3fd9d1579e6925fc7c1cb3a3fc851323c6b395f590c61"}
Jul 09 12:56:55.964253 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:56:55.964199    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724" podStartSLOduration=1.9641828879999998 podStartE2EDuration="1.964182888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:56:55.963259477 +0000 UTC m=+1973.557883519" watchObservedRunningTime="2026-07-09 12:56:55.964182888 +0000 UTC m=+1973.558806931"
Jul 09 12:57:05.078840 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:05.078794    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:57:05.078840 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:05.078851    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:57:05.091400 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:05.091374    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:57:06.000343 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:06.000312    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:57:14.321674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:14.321636    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"]
Jul 09 12:57:16.004573 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.004535    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"]
Jul 09 12:57:16.005047 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.004763    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" podUID="9b587e81-0ade-4abc-9e79-295c1224e44a" containerName="maas-api" containerID="cri-o://503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976" gracePeriod=30
Jul 09 12:57:16.270018 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.269935    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:57:16.386588 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.386537    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jtwl\" (UniqueName: \"kubernetes.io/projected/9b587e81-0ade-4abc-9e79-295c1224e44a-kube-api-access-2jtwl\") pod \"9b587e81-0ade-4abc-9e79-295c1224e44a\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") "
Jul 09 12:57:16.386817 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.386717    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b587e81-0ade-4abc-9e79-295c1224e44a-openshift-service-ca\") pod \"9b587e81-0ade-4abc-9e79-295c1224e44a\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") "
Jul 09 12:57:16.386817 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.386744    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls\") pod \"9b587e81-0ade-4abc-9e79-295c1224e44a\" (UID: \"9b587e81-0ade-4abc-9e79-295c1224e44a\") "
Jul 09 12:57:16.387098 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.387067    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b587e81-0ade-4abc-9e79-295c1224e44a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b587e81-0ade-4abc-9e79-295c1224e44a" (UID: "9b587e81-0ade-4abc-9e79-295c1224e44a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:57:16.388749 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.388721    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b587e81-0ade-4abc-9e79-295c1224e44a" (UID: "9b587e81-0ade-4abc-9e79-295c1224e44a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:57:16.388863 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.388816    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b587e81-0ade-4abc-9e79-295c1224e44a-kube-api-access-2jtwl" (OuterVolumeSpecName: "kube-api-access-2jtwl") pod "9b587e81-0ade-4abc-9e79-295c1224e44a" (UID: "9b587e81-0ade-4abc-9e79-295c1224e44a"). InnerVolumeSpecName "kube-api-access-2jtwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:57:16.488071 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.488035    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jtwl\" (UniqueName: \"kubernetes.io/projected/9b587e81-0ade-4abc-9e79-295c1224e44a-kube-api-access-2jtwl\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:16.488071 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.488068    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b587e81-0ade-4abc-9e79-295c1224e44a-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:16.488071 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:16.488082    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b587e81-0ade-4abc-9e79-295c1224e44a-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:17.048469 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.048371    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b587e81-0ade-4abc-9e79-295c1224e44a" containerID="503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976" exitCode=0
Jul 09 12:57:17.048469 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.048397    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"
Jul 09 12:57:17.049059 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.048477    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" event={"ID":"9b587e81-0ade-4abc-9e79-295c1224e44a","Type":"ContainerDied","Data":"503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976"}
Jul 09 12:57:17.049059 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.048552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b" event={"ID":"9b587e81-0ade-4abc-9e79-295c1224e44a","Type":"ContainerDied","Data":"a9ffe088ff38a37ccff6437073b6dbf02e90abb90b47c2526c67e185f05bf539"}
Jul 09 12:57:17.049059 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.048584    2574 scope.go:117] "RemoveContainer" containerID="503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976"
Jul 09 12:57:17.057658 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.057641    2574 scope.go:117] "RemoveContainer" containerID="503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976"
Jul 09 12:57:17.057975 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:57:17.057954    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976\": container with ID starting with 503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976 not found: ID does not exist" containerID="503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976"
Jul 09 12:57:17.058057 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.057982    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976"} err="failed to get container status \"503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976\": rpc error: code = NotFound desc = could not find container \"503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976\": container with ID starting with 503d628ae5553be1692bce5a59284ccc9c2802d4e09b405c5b578fba4b416976 not found: ID does not exist"
Jul 09 12:57:17.091460 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.091422    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"]
Jul 09 12:57:17.094733 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:17.094709    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-39792d-f4769fccd-b2b7b"]
Jul 09 12:57:18.971840 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:18.971801    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b587e81-0ade-4abc-9e79-295c1224e44a" path="/var/lib/kubelet/pods/9b587e81-0ade-4abc-9e79-295c1224e44a/volumes"
Jul 09 12:57:29.342179 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.342131    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724" podUID="62f95e95-b0d1-4df8-bd54-dcdf4f80acac" containerName="main" containerID="cri-o://1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24" gracePeriod=45
Jul 09 12:57:29.606492 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.606426    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:57:29.617170 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617148    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tls-certs\") pod \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") "
Jul 09 12:57:29.617311 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617191    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rw2v\" (UniqueName: \"kubernetes.io/projected/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-kube-api-access-2rw2v\") pod \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") "
Jul 09 12:57:29.617311 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617221    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tmp-dir\") pod \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") "
Jul 09 12:57:29.617311 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617297    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-dshm\") pod \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") "
Jul 09 12:57:29.617430 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617318    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-model-cache\") pod \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") "
Jul 09 12:57:29.617430 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617372    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-home\") pod \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\" (UID: \"62f95e95-b0d1-4df8-bd54-dcdf4f80acac\") "
Jul 09 12:57:29.617548 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617514    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "62f95e95-b0d1-4df8-bd54-dcdf4f80acac" (UID: "62f95e95-b0d1-4df8-bd54-dcdf4f80acac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:57:29.617743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617716    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-model-cache" (OuterVolumeSpecName: "model-cache") pod "62f95e95-b0d1-4df8-bd54-dcdf4f80acac" (UID: "62f95e95-b0d1-4df8-bd54-dcdf4f80acac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:57:29.617743 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617735    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tmp-dir\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:29.617890 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.617739    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-home" (OuterVolumeSpecName: "home") pod "62f95e95-b0d1-4df8-bd54-dcdf4f80acac" (UID: "62f95e95-b0d1-4df8-bd54-dcdf4f80acac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:57:29.619470 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.619443    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-dshm" (OuterVolumeSpecName: "dshm") pod "62f95e95-b0d1-4df8-bd54-dcdf4f80acac" (UID: "62f95e95-b0d1-4df8-bd54-dcdf4f80acac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:57:29.619570 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.619480    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "62f95e95-b0d1-4df8-bd54-dcdf4f80acac" (UID: "62f95e95-b0d1-4df8-bd54-dcdf4f80acac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:57:29.619570 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.619497    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-kube-api-access-2rw2v" (OuterVolumeSpecName: "kube-api-access-2rw2v") pod "62f95e95-b0d1-4df8-bd54-dcdf4f80acac" (UID: "62f95e95-b0d1-4df8-bd54-dcdf4f80acac"). InnerVolumeSpecName "kube-api-access-2rw2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:57:29.718239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.718206    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rw2v\" (UniqueName: \"kubernetes.io/projected/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-kube-api-access-2rw2v\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:29.718239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.718238    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-dshm\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:29.718451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.718251    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-model-cache\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:29.718451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.718261    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-home\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:29.718451 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:29.718269    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f95e95-b0d1-4df8-bd54-dcdf4f80acac-tls-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:57:30.125304 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.125270    2574 generic.go:356] "Generic (PLEG): container finished" podID="62f95e95-b0d1-4df8-bd54-dcdf4f80acac" containerID="1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24" exitCode=0
Jul 09 12:57:30.125493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.125315    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"
Jul 09 12:57:30.125493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.125320    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724" event={"ID":"62f95e95-b0d1-4df8-bd54-dcdf4f80acac","Type":"ContainerDied","Data":"1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24"}
Jul 09 12:57:30.125493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.125348    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724" event={"ID":"62f95e95-b0d1-4df8-bd54-dcdf4f80acac","Type":"ContainerDied","Data":"2c04b64b643c04f778f3fd9d1579e6925fc7c1cb3a3fc851323c6b395f590c61"}
Jul 09 12:57:30.125493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.125364    2574 scope.go:117] "RemoveContainer" containerID="1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24"
Jul 09 12:57:30.136165 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.136143    2574 scope.go:117] "RemoveContainer" containerID="1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24"
Jul 09 12:57:30.136545 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:57:30.136524    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24\": container with ID starting with 1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24 not found: ID does not exist" containerID="1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24"
Jul 09 12:57:30.136623 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.136558    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24"} err="failed to get container status \"1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24\": rpc error: code = NotFound desc = could not find container \"1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24\": container with ID starting with 1fec2b9cecf26e346939c9f34b11cae49d2282746b25cc122aec195101fabf24 not found: ID does not exist"
Jul 09 12:57:30.156780 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.156748    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"]
Jul 09 12:57:30.161142 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.161115    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-cb649b/test-model-cb649b-kserve-694fcb657c-46724"]
Jul 09 12:57:30.971478 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:30.971440    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f95e95-b0d1-4df8-bd54-dcdf4f80acac" path="/var/lib/kubelet/pods/62f95e95-b0d1-4df8-bd54-dcdf4f80acac/volumes"
Jul 09 12:57:58.961787 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:58.961746    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"]
Jul 09 12:57:58.962205 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:57:58.961975    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy" containerID="cri-o://4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059" gracePeriod=30
Jul 09 12:58:00.879941 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:00.879897    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"]
Jul 09 12:58:00.880534 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:00.880191    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" podUID="3724b5e2-89c3-4706-8872-72407200ccd1" containerName="maas-api" containerID="cri-o://2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7" gracePeriod=30
Jul 09 12:58:01.141289 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.141219    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:58:01.205109 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.205074    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3724b5e2-89c3-4706-8872-72407200ccd1-openshift-service-ca\") pod \"3724b5e2-89c3-4706-8872-72407200ccd1\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") "
Jul 09 12:58:01.205301 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.205151    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8fsk\" (UniqueName: \"kubernetes.io/projected/3724b5e2-89c3-4706-8872-72407200ccd1-kube-api-access-c8fsk\") pod \"3724b5e2-89c3-4706-8872-72407200ccd1\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") "
Jul 09 12:58:01.205301 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.205213    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls\") pod \"3724b5e2-89c3-4706-8872-72407200ccd1\" (UID: \"3724b5e2-89c3-4706-8872-72407200ccd1\") "
Jul 09 12:58:01.205448 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.205422    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3724b5e2-89c3-4706-8872-72407200ccd1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3724b5e2-89c3-4706-8872-72407200ccd1" (UID: "3724b5e2-89c3-4706-8872-72407200ccd1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:58:01.207327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.207300    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3724b5e2-89c3-4706-8872-72407200ccd1-kube-api-access-c8fsk" (OuterVolumeSpecName: "kube-api-access-c8fsk") pod "3724b5e2-89c3-4706-8872-72407200ccd1" (UID: "3724b5e2-89c3-4706-8872-72407200ccd1"). InnerVolumeSpecName "kube-api-access-c8fsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:01.207327 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.207318    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3724b5e2-89c3-4706-8872-72407200ccd1" (UID: "3724b5e2-89c3-4706-8872-72407200ccd1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:58:01.289148 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.289116    2574 generic.go:356] "Generic (PLEG): container finished" podID="3724b5e2-89c3-4706-8872-72407200ccd1" containerID="2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7" exitCode=0
Jul 09 12:58:01.289302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.289160    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"
Jul 09 12:58:01.289302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.289176    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" event={"ID":"3724b5e2-89c3-4706-8872-72407200ccd1","Type":"ContainerDied","Data":"2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7"}
Jul 09 12:58:01.289302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.289219    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg" event={"ID":"3724b5e2-89c3-4706-8872-72407200ccd1","Type":"ContainerDied","Data":"df4d8cd568a142d88a52270346b46967b18ea688a8cd2204cba427f15bbedf79"}
Jul 09 12:58:01.289302 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.289240    2574 scope.go:117] "RemoveContainer" containerID="2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7"
Jul 09 12:58:01.299328 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.299310    2574 scope.go:117] "RemoveContainer" containerID="2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7"
Jul 09 12:58:01.299623 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:01.299603    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7\": container with ID starting with 2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7 not found: ID does not exist" containerID="2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7"
Jul 09 12:58:01.299694 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.299637    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7"} err="failed to get container status \"2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7\": rpc error: code = NotFound desc = could not find container \"2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7\": container with ID starting with 2259672161c914c083f418b139b6e827231cb59df5e51b2caf53d672166b5ab7 not found: ID does not exist"
Jul 09 12:58:01.306146 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.306124    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3724b5e2-89c3-4706-8872-72407200ccd1-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:01.306146 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.306147    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3724b5e2-89c3-4706-8872-72407200ccd1-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:01.306293 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.306157    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8fsk\" (UniqueName: \"kubernetes.io/projected/3724b5e2-89c3-4706-8872-72407200ccd1-kube-api-access-c8fsk\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:01.319276 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.319246    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"]
Jul 09 12:58:01.336427 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:01.336395    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cb649b-69f9d8b88f-97zzg"]
Jul 09 12:58:02.971781 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:02.971748    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3724b5e2-89c3-4706-8872-72407200ccd1" path="/var/lib/kubelet/pods/3724b5e2-89c3-4706-8872-72407200ccd1/volumes"
Jul 09 12:58:04.227911 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.227842    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:58:04.308443 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.308407    2574 generic.go:356] "Generic (PLEG): container finished" podID="eb2086c1-0294-4845-a967-933635bc2f60" containerID="4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059" exitCode=0
Jul 09 12:58:04.308666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.308458    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"
Jul 09 12:58:04.308666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.308490    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" event={"ID":"eb2086c1-0294-4845-a967-933635bc2f60","Type":"ContainerDied","Data":"4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059"}
Jul 09 12:58:04.308666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.308557    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl" event={"ID":"eb2086c1-0294-4845-a967-933635bc2f60","Type":"ContainerDied","Data":"0d2701f49dfb3c4082fd675a625f6105ba922b0d92f88f81da9051889aff19b9"}
Jul 09 12:58:04.308666 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.308590    2574 scope.go:117] "RemoveContainer" containerID="4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059"
Jul 09 12:58:04.317617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.317598    2574 scope.go:117] "RemoveContainer" containerID="d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"
Jul 09 12:58:04.325649 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.325628    2574 scope.go:117] "RemoveContainer" containerID="4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059"
Jul 09 12:58:04.332245 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:04.325931    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059\": container with ID starting with 4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059 not found: ID does not exist" containerID="4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059"
Jul 09 12:58:04.332245 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.326311    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059"} err="failed to get container status \"4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059\": rpc error: code = NotFound desc = could not find container \"4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059\": container with ID starting with 4394d814a8f8d309540b051705f79feaec7669eed1d8b5971d9fffac50492059 not found: ID does not exist"
Jul 09 12:58:04.332245 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.326345    2574 scope.go:117] "RemoveContainer" containerID="d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"
Jul 09 12:58:04.332536 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:04.332500    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e\": container with ID starting with d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e not found: ID does not exist" containerID="d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"
Jul 09 12:58:04.332591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.332567    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e"} err="failed to get container status \"d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e\": rpc error: code = NotFound desc = could not find container \"d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e\": container with ID starting with d0462fb4870afb44b832c56989737f008a57592cb0d0adb75e351a4f3126ea2e not found: ID does not exist"
Jul 09 12:58:04.334920 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.334898    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-data\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335025 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.334945    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-envoy\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335025 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.334971    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-credential-socket\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335025 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335008    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2086c1-0294-4845-a967-933635bc2f60-istio-podinfo\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335132    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-istio-token\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335169    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-socket\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335194    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md9zw\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-kube-api-access-md9zw\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335239 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335225    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-certs\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335434 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335250    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2086c1-0294-4845-a967-933635bc2f60-istiod-ca-cert\") pod \"eb2086c1-0294-4845-a967-933635bc2f60\" (UID: \"eb2086c1-0294-4845-a967-933635bc2f60\") "
Jul 09 12:58:04.335434 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335286    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:04.335530 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335485    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:04.335617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335572    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:04.335708 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335586    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.335708 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335690    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.335828 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335760    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:04.335828 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.335816    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2086c1-0294-4845-a967-933635bc2f60-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:58:04.337378 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.337351    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb2086c1-0294-4845-a967-933635bc2f60-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:58:04.337378 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.337358    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:04.337734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.337660    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-kube-api-access-md9zw" (OuterVolumeSpecName: "kube-api-access-md9zw") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "kube-api-access-md9zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:04.337734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.337695    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb2086c1-0294-4845-a967-933635bc2f60" (UID: "eb2086c1-0294-4845-a967-933635bc2f60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:04.436519 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436478    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.436519 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436512    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.436519 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436523    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md9zw\" (UniqueName: \"kubernetes.io/projected/eb2086c1-0294-4845-a967-933635bc2f60-kube-api-access-md9zw\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.436811 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436535    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2086c1-0294-4845-a967-933635bc2f60-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.436811 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436543    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.436811 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436551    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2086c1-0294-4845-a967-933635bc2f60-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.436811 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.436558    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2086c1-0294-4845-a967-933635bc2f60-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:04.639671 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.639625    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"]
Jul 09 12:58:04.642362 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.642332    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-39792d-openshift-default-7b5d4b65df-rvfbl"]
Jul 09 12:58:04.972100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:04.972063    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb2086c1-0294-4845-a967-933635bc2f60" path="/var/lib/kubelet/pods/eb2086c1-0294-4845-a967-933635bc2f60/volumes"
Jul 09 12:58:08.929409 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.929373    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"]
Jul 09 12:58:08.929950 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.929933    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy"
Jul 09 12:58:08.929997 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.929954    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy"
Jul 09 12:58:08.930032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.929995    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b587e81-0ade-4abc-9e79-295c1224e44a" containerName="maas-api"
Jul 09 12:58:08.930032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930004    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b587e81-0ade-4abc-9e79-295c1224e44a" containerName="maas-api"
Jul 09 12:58:08.930032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930013    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f95e95-b0d1-4df8-bd54-dcdf4f80acac" containerName="main"
Jul 09 12:58:08.930032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930021    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f95e95-b0d1-4df8-bd54-dcdf4f80acac" containerName="main"
Jul 09 12:58:08.930151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930037    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3724b5e2-89c3-4706-8872-72407200ccd1" containerName="maas-api"
Jul 09 12:58:08.930151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930045    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3724b5e2-89c3-4706-8872-72407200ccd1" containerName="maas-api"
Jul 09 12:58:08.930151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930125    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3724b5e2-89c3-4706-8872-72407200ccd1" containerName="maas-api"
Jul 09 12:58:08.930151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930139    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy"
Jul 09 12:58:08.930151 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930149    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy"
Jul 09 12:58:08.930296 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930157    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f95e95-b0d1-4df8-bd54-dcdf4f80acac" containerName="main"
Jul 09 12:58:08.930296 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.930169    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b587e81-0ade-4abc-9e79-295c1224e44a" containerName="maas-api"
Jul 09 12:58:08.933549 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.933529    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:08.936900 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.936873    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-970533-openshift-default-dockercfg-p6qhr\""
Jul 09 12:58:08.964210 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:08.964174    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"]
Jul 09 12:58:09.080634 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-envoy\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.080871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-credential-socket\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.080871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080782    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-token\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.080871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-socket\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.080871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080858    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98e15d6-a6ea-43a9-be52-06688d2341ac-istiod-ca-cert\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.081083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-certs\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.081083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-podinfo\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.081083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-data\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.081083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.080946    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flv6z\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-kube-api-access-flv6z\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182354 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182243    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-certs\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182354 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-podinfo\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182354 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182333    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-data\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182354 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flv6z\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-kube-api-access-flv6z\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-envoy\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-credential-socket\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-token\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-socket\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182674 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98e15d6-a6ea-43a9-be52-06688d2341ac-istiod-ca-cert\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182977 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-certs\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.182977 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.182806    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-data\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.183083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.183015    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-credential-socket\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.183122 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.183094    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-socket\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.183248 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.183228    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98e15d6-a6ea-43a9-be52-06688d2341ac-istiod-ca-cert\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.184862 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.184845    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-envoy\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.185011 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.184994    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-podinfo\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.192581 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.192555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-token\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.193370 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.193349    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flv6z\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-kube-api-access-flv6z\") pod \"e2e-api-a-970533-openshift-default-546b499df7-hxrgh\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") " pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.246778 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.246736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:09.410474 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.410450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"]
Jul 09 12:58:09.413578 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:58:09.413549    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98e15d6_a6ea_43a9_be52_06688d2341ac.slice/crio-97a66bd52b8e9b7f244f64874894b8cab7c0fe6fb46bffef09ffba589f6060b7 WatchSource:0}: Error finding container 97a66bd52b8e9b7f244f64874894b8cab7c0fe6fb46bffef09ffba589f6060b7: Status 404 returned error can't find the container with id 97a66bd52b8e9b7f244f64874894b8cab7c0fe6fb46bffef09ffba589f6060b7
Jul 09 12:58:09.415914 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.415879    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:09.416033 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.415946    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:09.416033 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:09.415976    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:10.343828 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:10.343787    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" event={"ID":"a98e15d6-a6ea-43a9-be52-06688d2341ac","Type":"ContainerStarted","Data":"2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"}
Jul 09 12:58:10.344226 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:10.343850    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" event={"ID":"a98e15d6-a6ea-43a9-be52-06688d2341ac","Type":"ContainerStarted","Data":"97a66bd52b8e9b7f244f64874894b8cab7c0fe6fb46bffef09ffba589f6060b7"}
Jul 09 12:58:10.459493 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:10.459431    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podStartSLOduration=2.459412451 podStartE2EDuration="2.459412451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:58:10.456546225 +0000 UTC m=+2048.051170287" watchObservedRunningTime="2026-07-09 12:58:10.459412451 +0000 UTC m=+2048.054036493"
Jul 09 12:58:11.247424 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:11.247384    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:11.252503 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:11.252478    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:11.348734 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:11.348703    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:11.349761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:11.349736    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:12.668144 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.668109    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"]
Jul 09 12:58:12.668575 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.668532    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy"
Jul 09 12:58:12.668575 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.668544    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb2086c1-0294-4845-a967-933635bc2f60" containerName="istio-proxy"
Jul 09 12:58:12.671749 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.671733    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.675477 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.675456    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-970533-serving-cert\""
Jul 09 12:58:12.698236 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.698199    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"]
Jul 09 12:58:12.731444 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.731403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a3214b-d8d2-4599-8248-4ca9aa1f3215-maas-api-tls\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.731444 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.731449    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6rsj\" (UniqueName: \"kubernetes.io/projected/11a3214b-d8d2-4599-8248-4ca9aa1f3215-kube-api-access-l6rsj\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.731761 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.731536    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a3214b-d8d2-4599-8248-4ca9aa1f3215-openshift-service-ca\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.832487 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.832446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a3214b-d8d2-4599-8248-4ca9aa1f3215-openshift-service-ca\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.832750 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.832579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a3214b-d8d2-4599-8248-4ca9aa1f3215-maas-api-tls\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.832750 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.832598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6rsj\" (UniqueName: \"kubernetes.io/projected/11a3214b-d8d2-4599-8248-4ca9aa1f3215-kube-api-access-l6rsj\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.833182 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.833159    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a3214b-d8d2-4599-8248-4ca9aa1f3215-openshift-service-ca\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.835148 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.835126    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a3214b-d8d2-4599-8248-4ca9aa1f3215-maas-api-tls\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.854273 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.854240    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6rsj\" (UniqueName: \"kubernetes.io/projected/11a3214b-d8d2-4599-8248-4ca9aa1f3215-kube-api-access-l6rsj\") pod \"maas-api-e2e-api-a-970533-7b9b49cd88-lkplv\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:12.985767 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:12.985717    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:13.143488 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:13.143442    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"]
Jul 09 12:58:13.359230 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:13.359127    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv" event={"ID":"11a3214b-d8d2-4599-8248-4ca9aa1f3215","Type":"ContainerStarted","Data":"420dd2870cf803034197ced9566e393b9d2a3d7a6b43e3fc44c9d623b371bc18"}
Jul 09 12:58:14.366524 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:14.366483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv" event={"ID":"11a3214b-d8d2-4599-8248-4ca9aa1f3215","Type":"ContainerStarted","Data":"6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae"}
Jul 09 12:58:14.366952 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:14.366665    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:14.404321 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:14.404262    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv" podStartSLOduration=1.981119714 podStartE2EDuration="2.404245521s" totalImagesPullingTime="423.125807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:58:14.40172937 +0000 UTC m=+2051.996353407" watchObservedRunningTime="2026-07-09 12:58:14.404245521 +0000 UTC m=+2051.998869563"
Jul 09 12:58:20.377918 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:20.377884    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:21.418453 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.418411    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"]
Jul 09 12:58:21.422279 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.422255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.426608 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.426583    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6e7f0e-serving-cert\""
Jul 09 12:58:21.453708 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.449494    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"]
Jul 09 12:58:21.517799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.517732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7952689d-c2ea-4501-b502-5113b83a12d1-openshift-service-ca\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.517799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.517803    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.518100 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.517846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wk5p\" (UniqueName: \"kubernetes.io/projected/7952689d-c2ea-4501-b502-5113b83a12d1-kube-api-access-9wk5p\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.618725 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.618670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.618919 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.618766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wk5p\" (UniqueName: \"kubernetes.io/projected/7952689d-c2ea-4501-b502-5113b83a12d1-kube-api-access-9wk5p\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.618919 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:21.618829    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-serving-cert: secret "maas-api-e2e-api-b-6e7f0e-serving-cert" not found
Jul 09 12:58:21.618919 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:21.618897    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls podName:7952689d-c2ea-4501-b502-5113b83a12d1 nodeName:}" failed. No retries permitted until 2026-07-09 12:58:22.118877319 +0000 UTC m=+2059.713501352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls") pod "maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" (UID: "7952689d-c2ea-4501-b502-5113b83a12d1") : secret "maas-api-e2e-api-b-6e7f0e-serving-cert" not found
Jul 09 12:58:21.619048 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.618930    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7952689d-c2ea-4501-b502-5113b83a12d1-openshift-service-ca\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.619569 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.619549    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7952689d-c2ea-4501-b502-5113b83a12d1-openshift-service-ca\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:21.636904 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:21.636871    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wk5p\" (UniqueName: \"kubernetes.io/projected/7952689d-c2ea-4501-b502-5113b83a12d1-kube-api-access-9wk5p\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:22.124001 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:22.123916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:22.126537 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:22.126502    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls\") pod \"maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:22.333580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:22.333524    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:22.576172 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:22.568649    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"]
Jul 09 12:58:22.580388 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:58:22.580276    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7952689d_c2ea_4501_b502_5113b83a12d1.slice/crio-8e3ad3acac79ad893e0a2d64db9ad1c6c72a98c11d45669208f8724abe4fdbe1 WatchSource:0}: Error finding container 8e3ad3acac79ad893e0a2d64db9ad1c6c72a98c11d45669208f8724abe4fdbe1: Status 404 returned error can't find the container with id 8e3ad3acac79ad893e0a2d64db9ad1c6c72a98c11d45669208f8724abe4fdbe1
Jul 09 12:58:23.422837 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:23.422800    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" event={"ID":"7952689d-c2ea-4501-b502-5113b83a12d1","Type":"ContainerStarted","Data":"10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440"}
Jul 09 12:58:23.423126 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:23.422862    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" event={"ID":"7952689d-c2ea-4501-b502-5113b83a12d1","Type":"ContainerStarted","Data":"8e3ad3acac79ad893e0a2d64db9ad1c6c72a98c11d45669208f8724abe4fdbe1"}
Jul 09 12:58:23.423126 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:23.422890    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:23.471858 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:23.471795    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" podStartSLOduration=2.115635405 podStartE2EDuration="2.471772882s" totalImagesPullingTime="356.137477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:58:23.470619959 +0000 UTC m=+2061.065244006" watchObservedRunningTime="2026-07-09 12:58:23.471772882 +0000 UTC m=+2061.066396928"
Jul 09 12:58:24.429287 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:24.429254    2574 generic.go:356] "Generic (PLEG): container finished" podID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerID="2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d" exitCode=0
Jul 09 12:58:24.429754 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:24.429337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" event={"ID":"a98e15d6-a6ea-43a9-be52-06688d2341ac","Type":"ContainerDied","Data":"2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"}
Jul 09 12:58:24.429894 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:24.429875    2574 scope.go:117] "RemoveContainer" containerID="2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"
Jul 09 12:58:24.430851 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:24.430825    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:24.430918 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:24.430889    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:24.430967 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:24.430921    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:25.438651 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:25.438546    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" event={"ID":"a98e15d6-a6ea-43a9-be52-06688d2341ac","Type":"ContainerStarted","Data":"167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47"}
Jul 09 12:58:26.249355 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:26.249316    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:26.249797 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:26.249771    2574 patch_prober.go:28] interesting pod/e2e-api-a-970533-openshift-default-546b499df7-hxrgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 09 12:58:26.250000 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:26.249971    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 09 12:58:26.348695 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:26.348625    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:27.248235 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:27.248195    2574 patch_prober.go:28] interesting pod/e2e-api-a-970533-openshift-default-546b499df7-hxrgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 09 12:58:27.248778 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:27.248267    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 09 12:58:28.248300 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:28.248256    2574 patch_prober.go:28] interesting pod/e2e-api-a-970533-openshift-default-546b499df7-hxrgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 09 12:58:28.249010 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:28.248975    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 09 12:58:29.247935 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:29.247885    2574 patch_prober.go:28] interesting pod/e2e-api-a-970533-openshift-default-546b499df7-hxrgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 09 12:58:29.248243 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:29.248209    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 09 12:58:29.438144 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:29.438106    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:30.251866 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:30.251826    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:30.252913 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:30.252888    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:34.973902 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:34.973870    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"]
Jul 09 12:58:34.974495 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:34.974121    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv" podUID="11a3214b-d8d2-4599-8248-4ca9aa1f3215" containerName="maas-api" containerID="cri-o://6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae" gracePeriod=30
Jul 09 12:58:35.231789 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.231718    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:35.348442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.348410    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a3214b-d8d2-4599-8248-4ca9aa1f3215-openshift-service-ca\") pod \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") "
Jul 09 12:58:35.348635 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.348488    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6rsj\" (UniqueName: \"kubernetes.io/projected/11a3214b-d8d2-4599-8248-4ca9aa1f3215-kube-api-access-l6rsj\") pod \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") "
Jul 09 12:58:35.348635 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.348514    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a3214b-d8d2-4599-8248-4ca9aa1f3215-maas-api-tls\") pod \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\" (UID: \"11a3214b-d8d2-4599-8248-4ca9aa1f3215\") "
Jul 09 12:58:35.348916 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.348886    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a3214b-d8d2-4599-8248-4ca9aa1f3215-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11a3214b-d8d2-4599-8248-4ca9aa1f3215" (UID: "11a3214b-d8d2-4599-8248-4ca9aa1f3215"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:58:35.350671 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.350641    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a3214b-d8d2-4599-8248-4ca9aa1f3215-kube-api-access-l6rsj" (OuterVolumeSpecName: "kube-api-access-l6rsj") pod "11a3214b-d8d2-4599-8248-4ca9aa1f3215" (UID: "11a3214b-d8d2-4599-8248-4ca9aa1f3215"). InnerVolumeSpecName "kube-api-access-l6rsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:35.350806 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.350699    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a3214b-d8d2-4599-8248-4ca9aa1f3215-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11a3214b-d8d2-4599-8248-4ca9aa1f3215" (UID: "11a3214b-d8d2-4599-8248-4ca9aa1f3215"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:58:35.449954 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.449914    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a3214b-d8d2-4599-8248-4ca9aa1f3215-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:35.449954 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.449945    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6rsj\" (UniqueName: \"kubernetes.io/projected/11a3214b-d8d2-4599-8248-4ca9aa1f3215-kube-api-access-l6rsj\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:35.449954 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.449955    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a3214b-d8d2-4599-8248-4ca9aa1f3215-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:35.502759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.502655    2574 generic.go:356] "Generic (PLEG): container finished" podID="11a3214b-d8d2-4599-8248-4ca9aa1f3215" containerID="6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae" exitCode=0
Jul 09 12:58:35.502759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.502737    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"
Jul 09 12:58:35.502759 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.502748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv" event={"ID":"11a3214b-d8d2-4599-8248-4ca9aa1f3215","Type":"ContainerDied","Data":"6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae"}
Jul 09 12:58:35.503021 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.502805    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv" event={"ID":"11a3214b-d8d2-4599-8248-4ca9aa1f3215","Type":"ContainerDied","Data":"420dd2870cf803034197ced9566e393b9d2a3d7a6b43e3fc44c9d623b371bc18"}
Jul 09 12:58:35.503021 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.502829    2574 scope.go:117] "RemoveContainer" containerID="6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae"
Jul 09 12:58:35.512839 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.512815    2574 scope.go:117] "RemoveContainer" containerID="6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae"
Jul 09 12:58:35.513104 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:35.513086    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae\": container with ID starting with 6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae not found: ID does not exist" containerID="6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae"
Jul 09 12:58:35.513169 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.513115    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae"} err="failed to get container status \"6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae\": rpc error: code = NotFound desc = could not find container \"6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae\": container with ID starting with 6c2d037539a930eae59c31d1b4000d3d8b52e666859076a3f1ef40c327caacae not found: ID does not exist"
Jul 09 12:58:35.526086 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.526055    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"]
Jul 09 12:58:35.530622 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:35.530597    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-970533-7b9b49cd88-lkplv"]
Jul 09 12:58:36.971082 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:36.971039    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a3214b-d8d2-4599-8248-4ca9aa1f3215" path="/var/lib/kubelet/pods/11a3214b-d8d2-4599-8248-4ca9aa1f3215/volumes"
Jul 09 12:58:42.919045 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:42.919007    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"]
Jul 09 12:58:42.919444 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:42.919221    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy" containerID="cri-o://167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47" gracePeriod=30
Jul 09 12:58:44.182332 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.182294    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"]
Jul 09 12:58:44.182871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.182546    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" podUID="7952689d-c2ea-4501-b502-5113b83a12d1" containerName="maas-api" containerID="cri-o://10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440" gracePeriod=30
Jul 09 12:58:44.445953 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.445924    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:44.538162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.538097    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7952689d-c2ea-4501-b502-5113b83a12d1-openshift-service-ca\") pod \"7952689d-c2ea-4501-b502-5113b83a12d1\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") "
Jul 09 12:58:44.538162 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.538166    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wk5p\" (UniqueName: \"kubernetes.io/projected/7952689d-c2ea-4501-b502-5113b83a12d1-kube-api-access-9wk5p\") pod \"7952689d-c2ea-4501-b502-5113b83a12d1\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") "
Jul 09 12:58:44.538434 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.538244    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls\") pod \"7952689d-c2ea-4501-b502-5113b83a12d1\" (UID: \"7952689d-c2ea-4501-b502-5113b83a12d1\") "
Jul 09 12:58:44.538606 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.538576    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7952689d-c2ea-4501-b502-5113b83a12d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7952689d-c2ea-4501-b502-5113b83a12d1" (UID: "7952689d-c2ea-4501-b502-5113b83a12d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:58:44.540699 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.540653    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7952689d-c2ea-4501-b502-5113b83a12d1-kube-api-access-9wk5p" (OuterVolumeSpecName: "kube-api-access-9wk5p") pod "7952689d-c2ea-4501-b502-5113b83a12d1" (UID: "7952689d-c2ea-4501-b502-5113b83a12d1"). InnerVolumeSpecName "kube-api-access-9wk5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:44.540818 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.540706    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7952689d-c2ea-4501-b502-5113b83a12d1" (UID: "7952689d-c2ea-4501-b502-5113b83a12d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:58:44.554375 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.554337    2574 generic.go:356] "Generic (PLEG): container finished" podID="7952689d-c2ea-4501-b502-5113b83a12d1" containerID="10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440" exitCode=0
Jul 09 12:58:44.554521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.554388    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"
Jul 09 12:58:44.554521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.554424    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" event={"ID":"7952689d-c2ea-4501-b502-5113b83a12d1","Type":"ContainerDied","Data":"10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440"}
Jul 09 12:58:44.554521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.554485    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" event={"ID":"7952689d-c2ea-4501-b502-5113b83a12d1","Type":"ContainerDied","Data":"8e3ad3acac79ad893e0a2d64db9ad1c6c72a98c11d45669208f8724abe4fdbe1"}
Jul 09 12:58:44.554521 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.554507    2574 scope.go:117] "RemoveContainer" containerID="10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440"
Jul 09 12:58:44.568737 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.568716    2574 scope.go:117] "RemoveContainer" containerID="10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440"
Jul 09 12:58:44.569059 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:44.569036    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440\": container with ID starting with 10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440 not found: ID does not exist" containerID="10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440"
Jul 09 12:58:44.569140 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.569064    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440"} err="failed to get container status \"10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440\": rpc error: code = NotFound desc = could not find container \"10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440\": container with ID starting with 10007b9231a4184c9589d9f8eac7df5d581b412fcd1b5782eb45608cedc12440 not found: ID does not exist"
Jul 09 12:58:44.591905 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.591867    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"]
Jul 09 12:58:44.596073 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.596045    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf"]
Jul 09 12:58:44.639848 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.639812    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7952689d-c2ea-4501-b502-5113b83a12d1-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:44.639848 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.639840    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wk5p\" (UniqueName: \"kubernetes.io/projected/7952689d-c2ea-4501-b502-5113b83a12d1-kube-api-access-9wk5p\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:44.639848 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.639851    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7952689d-c2ea-4501-b502-5113b83a12d1-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:44.971617 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:44.971584    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7952689d-c2ea-4501-b502-5113b83a12d1" path="/var/lib/kubelet/pods/7952689d-c2ea-4501-b502-5113b83a12d1/volumes"
Jul 09 12:58:45.254467 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:45.254368    2574 patch_prober.go:28] interesting pod/e2e-api-a-970533-openshift-default-546b499df7-hxrgh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 09 12:58:45.254467 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:45.254446    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:58:47.431028 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:47.430975    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6e7f0e-57f769596c-q2qcf" podUID="7952689d-c2ea-4501-b502-5113b83a12d1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.66:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 09 12:58:48.173402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.173372    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:48.274586 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274548    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-token\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.274816 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274630    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-podinfo\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.274816 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274658    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-envoy\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.274816 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274745    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-socket\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.274816 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274772    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-certs\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.274816 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274803    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98e15d6-a6ea-43a9-be52-06688d2341ac-istiod-ca-cert\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.275083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274856    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-credential-socket\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.275083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274892    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flv6z\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-kube-api-access-flv6z\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.275083 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.274925    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-data\") pod \"a98e15d6-a6ea-43a9-be52-06688d2341ac\" (UID: \"a98e15d6-a6ea-43a9-be52-06688d2341ac\") "
Jul 09 12:58:48.275238 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.275048    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:48.275238 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.275109    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:48.275355 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.275278    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:48.275355 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.275282    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.275355 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.275337    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.275574 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.275542    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98e15d6-a6ea-43a9-be52-06688d2341ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:58:48.276066 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.276037    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:48.277042 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.277020    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:58:48.277133 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.277035    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:58:48.277262 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.277245    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:48.277353 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.277332    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-kube-api-access-flv6z" (OuterVolumeSpecName: "kube-api-access-flv6z") pod "a98e15d6-a6ea-43a9-be52-06688d2341ac" (UID: "a98e15d6-a6ea-43a9-be52-06688d2341ac"). InnerVolumeSpecName "kube-api-access-flv6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:58:48.376122 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376085    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98e15d6-a6ea-43a9-be52-06688d2341ac-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.376122 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376115    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.376122 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376124    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flv6z\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-kube-api-access-flv6z\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.376386 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376135    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.376386 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376143    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.376386 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376151    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.376386 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.376158    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98e15d6-a6ea-43a9-be52-06688d2341ac-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 12:58:48.578260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.578165    2574 generic.go:356] "Generic (PLEG): container finished" podID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerID="167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47" exitCode=0
Jul 09 12:58:48.578260 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.578218    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"
Jul 09 12:58:48.578797 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.578255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" event={"ID":"a98e15d6-a6ea-43a9-be52-06688d2341ac","Type":"ContainerDied","Data":"167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47"}
Jul 09 12:58:48.578797 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.578325    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh" event={"ID":"a98e15d6-a6ea-43a9-be52-06688d2341ac","Type":"ContainerDied","Data":"97a66bd52b8e9b7f244f64874894b8cab7c0fe6fb46bffef09ffba589f6060b7"}
Jul 09 12:58:48.578797 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.578356    2574 scope.go:117] "RemoveContainer" containerID="167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47"
Jul 09 12:58:48.588259 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.588242    2574 scope.go:117] "RemoveContainer" containerID="2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"
Jul 09 12:58:48.596534 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.596513    2574 scope.go:117] "RemoveContainer" containerID="167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47"
Jul 09 12:58:48.596803 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:48.596782    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47\": container with ID starting with 167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47 not found: ID does not exist" containerID="167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47"
Jul 09 12:58:48.596852 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.596813    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47"} err="failed to get container status \"167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47\": rpc error: code = NotFound desc = could not find container \"167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47\": container with ID starting with 167d99e71bd81a7d74fcc620eef02e86a19f45f43fb563cd038e486039effd47 not found: ID does not exist"
Jul 09 12:58:48.596852 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.596833    2574 scope.go:117] "RemoveContainer" containerID="2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"
Jul 09 12:58:48.597054 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:58:48.597034    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d\": container with ID starting with 2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d not found: ID does not exist" containerID="2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"
Jul 09 12:58:48.597112 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.597065    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d"} err="failed to get container status \"2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d\": rpc error: code = NotFound desc = could not find container \"2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d\": container with ID starting with 2ebd1c727c99c82091e55cb63c6e6fdf5c9475096fb9d8a8c7f3b03c78d1f00d not found: ID does not exist"
Jul 09 12:58:48.602851 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.602830    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"]
Jul 09 12:58:48.610426 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.610406    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-970533-openshift-default-546b499df7-hxrgh"]
Jul 09 12:58:48.971591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:48.971558    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" path="/var/lib/kubelet/pods/a98e15d6-a6ea-43a9-be52-06688d2341ac/volumes"
Jul 09 12:58:52.273391 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273351    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"]
Jul 09 12:58:52.273956 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273939    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a3214b-d8d2-4599-8248-4ca9aa1f3215" containerName="maas-api"
Jul 09 12:58:52.273956 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273958    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a3214b-d8d2-4599-8248-4ca9aa1f3215" containerName="maas-api"
Jul 09 12:58:52.274063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273972    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7952689d-c2ea-4501-b502-5113b83a12d1" containerName="maas-api"
Jul 09 12:58:52.274063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273981    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7952689d-c2ea-4501-b502-5113b83a12d1" containerName="maas-api"
Jul 09 12:58:52.274063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273991    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy"
Jul 09 12:58:52.274063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.273999    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy"
Jul 09 12:58:52.274063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.274024    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy"
Jul 09 12:58:52.274063 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.274033    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy"
Jul 09 12:58:52.274248 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.274113    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy"
Jul 09 12:58:52.274248 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.274127    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a3214b-d8d2-4599-8248-4ca9aa1f3215" containerName="maas-api"
Jul 09 12:58:52.274248 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.274140    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7952689d-c2ea-4501-b502-5113b83a12d1" containerName="maas-api"
Jul 09 12:58:52.278877 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.278857    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.282255 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.282236    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-83cace-openshift-default-dockercfg-ds9tm\""
Jul 09 12:58:52.292558 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.292534    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"]
Jul 09 12:58:52.412155 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-certs\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412155 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-podinfo\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-socket\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-data\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-envoy\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412385 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412315    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7sjf\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-kube-api-access-w7sjf\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412596 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-token\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412596 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412472    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-credential-socket\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.412596 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.412513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istiod-ca-cert\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.513599 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istiod-ca-cert\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.513814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513663    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-certs\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.513814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513719    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-podinfo\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.513814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-socket\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.513814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513773    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-data\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.513814 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513808    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-envoy\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7sjf\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-kube-api-access-w7sjf\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-token\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.513909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-credential-socket\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514097 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.514067    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-certs\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514294 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.514125    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-socket\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514294 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.514204    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-credential-socket\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514407 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.514307    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istiod-ca-cert\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.514407 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.514354    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-data\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.516344 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.516314    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-envoy\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.516602 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.516584    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-podinfo\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.526558 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.526489    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-token\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.526668 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.526555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7sjf\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-kube-api-access-w7sjf\") pod \"e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") " pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.589334 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.589292    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:52.741495 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.741468    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"]
Jul 09 12:58:52.742906 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:58:52.742877    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ef76ef_595f_4724_9e06_26bc50ae9ed2.slice/crio-8863f6410cd780cc6d0d67beb44e7d8cd3e3350a7c2791652046ef1b0986352d WatchSource:0}: Error finding container 8863f6410cd780cc6d0d67beb44e7d8cd3e3350a7c2791652046ef1b0986352d: Status 404 returned error can't find the container with id 8863f6410cd780cc6d0d67beb44e7d8cd3e3350a7c2791652046ef1b0986352d
Jul 09 12:58:52.745032 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.745000    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:52.745127 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.745070    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:52.745127 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:52.745098    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:58:53.609129 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:53.609092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" event={"ID":"13ef76ef-595f-4724-9e06-26bc50ae9ed2","Type":"ContainerStarted","Data":"ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"}
Jul 09 12:58:53.609620 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:53.609158    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" event={"ID":"13ef76ef-595f-4724-9e06-26bc50ae9ed2","Type":"ContainerStarted","Data":"8863f6410cd780cc6d0d67beb44e7d8cd3e3350a7c2791652046ef1b0986352d"}
Jul 09 12:58:53.637015 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:53.636962    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" podStartSLOduration=1.636945232 podStartE2EDuration="1.636945232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:58:53.63525335 +0000 UTC m=+2091.229877413" watchObservedRunningTime="2026-07-09 12:58:53.636945232 +0000 UTC m=+2091.231569276"
Jul 09 12:58:54.590402 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:54.590368    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:54.595634 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:54.595604    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:54.614333 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:54.614306    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:54.615161 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:54.615143    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:58:56.061928 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.061882    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"]
Jul 09 12:58:56.062569 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.062552    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a98e15d6-a6ea-43a9-be52-06688d2341ac" containerName="istio-proxy"
Jul 09 12:58:56.066192 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.066167    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.072919 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.072894    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-83cace-serving-cert\""
Jul 09 12:58:56.126636 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.126606    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"]
Jul 09 12:58:56.248965 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.248930    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fxxx\" (UniqueName: \"kubernetes.io/projected/3d5e9483-7226-4fab-abf3-d305748b7052-kube-api-access-5fxxx\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.248965 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.248971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9483-7226-4fab-abf3-d305748b7052-openshift-service-ca\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.249213 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.249027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5e9483-7226-4fab-abf3-d305748b7052-maas-api-tls\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.350068 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.349959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fxxx\" (UniqueName: \"kubernetes.io/projected/3d5e9483-7226-4fab-abf3-d305748b7052-kube-api-access-5fxxx\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.350068 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.350007    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9483-7226-4fab-abf3-d305748b7052-openshift-service-ca\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.350268 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.350144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5e9483-7226-4fab-abf3-d305748b7052-maas-api-tls\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.350784 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.350755    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9483-7226-4fab-abf3-d305748b7052-openshift-service-ca\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.352643 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.352618    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5e9483-7226-4fab-abf3-d305748b7052-maas-api-tls\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.378163 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.378130    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fxxx\" (UniqueName: \"kubernetes.io/projected/3d5e9483-7226-4fab-abf3-d305748b7052-kube-api-access-5fxxx\") pod \"maas-api-e2e-shared-a-83cace-758546c569-q78sg\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.677871 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.677767    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:56.837089 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:56.837063    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"]
Jul 09 12:58:56.838581 ip-10-0-140-237 kubenswrapper[2574]: W0709 12:58:56.838541    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5e9483_7226_4fab_abf3_d305748b7052.slice/crio-c43d82d42956ed3ce606c949b016d0d6fe9d5bb2fa525d9bab6192ece384d7c0 WatchSource:0}: Error finding container c43d82d42956ed3ce606c949b016d0d6fe9d5bb2fa525d9bab6192ece384d7c0: Status 404 returned error can't find the container with id c43d82d42956ed3ce606c949b016d0d6fe9d5bb2fa525d9bab6192ece384d7c0
Jul 09 12:58:57.632279 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:57.632240    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg" event={"ID":"3d5e9483-7226-4fab-abf3-d305748b7052","Type":"ContainerStarted","Data":"e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c"}
Jul 09 12:58:57.632713 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:57.632296    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg" event={"ID":"3d5e9483-7226-4fab-abf3-d305748b7052","Type":"ContainerStarted","Data":"c43d82d42956ed3ce606c949b016d0d6fe9d5bb2fa525d9bab6192ece384d7c0"}
Jul 09 12:58:57.632713 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:57.632358    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:58:57.664785 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:58:57.664729    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg" podStartSLOduration=2.3197414 podStartE2EDuration="2.664713269s" totalImagesPullingTime="344.971869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:58:57.662151464 +0000 UTC m=+2095.256775527" watchObservedRunningTime="2026-07-09 12:58:57.664713269 +0000 UTC m=+2095.259337312"
Jul 09 12:59:03.072729 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:03.072700    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:59:03.076206 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:03.076176    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 12:59:03.649397 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:03.649357    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 12:59:04.507662 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.507612    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"]
Jul 09 12:59:04.512648 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.512625    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.517915 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.517891    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-d5cca2-serving-cert\""
Jul 09 12:59:04.523669 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.523638    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.523789 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.523755    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.523870 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.523854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpfpc\" (UniqueName: \"kubernetes.io/projected/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-kube-api-access-kpfpc\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.540235 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.540197    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"]
Jul 09 12:59:04.624555 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.624517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfpc\" (UniqueName: \"kubernetes.io/projected/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-kube-api-access-kpfpc\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.624807 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.624563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.624807 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.624602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.624807 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:59:04.624728    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-serving-cert: secret "maas-api-e2e-shared-b-d5cca2-serving-cert" not found
Jul 09 12:59:04.624807 ip-10-0-140-237 kubenswrapper[2574]: E0709 12:59:04.624783    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls podName:a0e6497b-95ec-4bae-af36-6ab3d0ce684a nodeName:}" failed. No retries permitted until 2026-07-09 12:59:05.124767364 +0000 UTC m=+2102.719391384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls") pod "maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" (UID: "a0e6497b-95ec-4bae-af36-6ab3d0ce684a") : secret "maas-api-e2e-shared-b-d5cca2-serving-cert" not found
Jul 09 12:59:04.625270 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.625251    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:04.662704 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:04.661815    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpfpc\" (UniqueName: \"kubernetes.io/projected/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-kube-api-access-kpfpc\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:05.129296 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:05.129259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:05.131843 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:05.131815    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls\") pod \"maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:05.423646 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:05.423553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:05.614966 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:05.614174    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"]
Jul 09 12:59:05.680580 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:05.680484    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" event={"ID":"a0e6497b-95ec-4bae-af36-6ab3d0ce684a","Type":"ContainerStarted","Data":"ce6d261420f52f6cc772e4e6db61cf7c147e72695c6b688017575ad376aa48c7"}
Jul 09 12:59:06.687161 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:06.687123    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" event={"ID":"a0e6497b-95ec-4bae-af36-6ab3d0ce684a","Type":"ContainerStarted","Data":"1d1af852fe49b2f413e8a6d81f25994b4f7b029c9ca55f19c74427a83849db2f"}
Jul 09 12:59:06.687591 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:06.687254    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:06.722442 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:06.722372    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" podStartSLOduration=2.373550384 podStartE2EDuration="2.72234773s" totalImagesPullingTime="348.797346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:59:06.719136747 +0000 UTC m=+2104.313760792" watchObservedRunningTime="2026-07-09 12:59:06.72234773 +0000 UTC m=+2104.316971778"
Jul 09 12:59:08.699449 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:08.699418    2574 generic.go:356] "Generic (PLEG): container finished" podID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerID="ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799" exitCode=0
Jul 09 12:59:08.699943 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:08.699487    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" event={"ID":"13ef76ef-595f-4724-9e06-26bc50ae9ed2","Type":"ContainerDied","Data":"ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"}
Jul 09 12:59:08.699991 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:08.699967    2574 scope.go:117] "RemoveContainer" containerID="ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"
Jul 09 12:59:08.700993 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:08.700966    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:59:08.701068 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:08.701035    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:59:08.701109 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:08.701065    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 12:59:09.614777 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:09.614746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:59:09.706670 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:09.706631    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" event={"ID":"13ef76ef-595f-4724-9e06-26bc50ae9ed2","Type":"ContainerStarted","Data":"a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6"}
Jul 09 12:59:10.589770 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:10.589730    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:59:10.590780 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:10.590756    2574 patch_prober.go:28] interesting pod/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 09 12:59:10.590840 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:10.590807    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 09 12:59:11.590229 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:11.590181    2574 patch_prober.go:28] interesting pod/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 09 12:59:11.590633 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:11.590256    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 09 12:59:12.590121 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:12.590075    2574 patch_prober.go:28] interesting pod/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 09 12:59:12.590316 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:12.590153    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 09 12:59:12.698452 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:12.698420    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 12:59:13.590315 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:13.590269    2574 patch_prober.go:28] interesting pod/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 09 12:59:13.590760 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:13.590355    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 09 12:59:14.594381 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:14.594348    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:59:14.594799 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:14.594626    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 12:59:14.595491 ip-10-0-140-237 kubenswrapper[2574]: I0709 12:59:14.595472    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 13:04:03.103506 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:04:03.103474    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 13:04:03.108059 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:04:03.108034    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 13:05:27.896227 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:27.896174    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7471-140089823955120"]
Jul 09 13:05:27.899928 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:27.899905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:27.902651 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:27.902634    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-7bp7g\""
Jul 09 13:05:27.907258 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:27.907237    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7471-140089823955120"]
Jul 09 13:05:28.049578 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.049535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg4sb\" (UniqueName: \"kubernetes.io/projected/18d89028-2156-4bb4-9a4a-70a83819cef7-kube-api-access-wg4sb\") pod \"test-curl-7471-140089823955120\" (UID: \"18d89028-2156-4bb4-9a4a-70a83819cef7\") " pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:28.150982 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.150888    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg4sb\" (UniqueName: \"kubernetes.io/projected/18d89028-2156-4bb4-9a4a-70a83819cef7-kube-api-access-wg4sb\") pod \"test-curl-7471-140089823955120\" (UID: \"18d89028-2156-4bb4-9a4a-70a83819cef7\") " pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:28.159810 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.159779    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg4sb\" (UniqueName: \"kubernetes.io/projected/18d89028-2156-4bb4-9a4a-70a83819cef7-kube-api-access-wg4sb\") pod \"test-curl-7471-140089823955120\" (UID: \"18d89028-2156-4bb4-9a4a-70a83819cef7\") " pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:28.210407 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.210364    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:28.352088 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.352061    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7471-140089823955120"]
Jul 09 13:05:28.354812 ip-10-0-140-237 kubenswrapper[2574]: W0709 13:05:28.354779    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18d89028_2156_4bb4_9a4a_70a83819cef7.slice/crio-7918c338a47f1ba9a954169b7cd083e6544eb5ff6800f4ac0dd7189742fdfe37 WatchSource:0}: Error finding container 7918c338a47f1ba9a954169b7cd083e6544eb5ff6800f4ac0dd7189742fdfe37: Status 404 returned error can't find the container with id 7918c338a47f1ba9a954169b7cd083e6544eb5ff6800f4ac0dd7189742fdfe37
Jul 09 13:05:28.356051 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.356032    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 13:05:28.701034 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:28.700988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7471-140089823955120" event={"ID":"18d89028-2156-4bb4-9a4a-70a83819cef7","Type":"ContainerStarted","Data":"7918c338a47f1ba9a954169b7cd083e6544eb5ff6800f4ac0dd7189742fdfe37"}
Jul 09 13:05:29.707594 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:29.707551    2574 generic.go:356] "Generic (PLEG): container finished" podID="18d89028-2156-4bb4-9a4a-70a83819cef7" containerID="e0f335ed0fa0250260bbfc9c7f816b6bdd653e9220111f6c8646e7e8d85f4cb1" exitCode=0
Jul 09 13:05:29.708034 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:29.707601    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7471-140089823955120" event={"ID":"18d89028-2156-4bb4-9a4a-70a83819cef7","Type":"ContainerDied","Data":"e0f335ed0fa0250260bbfc9c7f816b6bdd653e9220111f6c8646e7e8d85f4cb1"}
Jul 09 13:05:30.865127 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:30.865095    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:30.907533 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:30.907498    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7471-140089823955120_18d89028-2156-4bb4-9a4a-70a83819cef7/test-curl-7471-140089823955120/0.log"
Jul 09 13:05:30.934873 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:30.934843    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7471-140089823955120"]
Jul 09 13:05:30.940526 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:30.940495    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7471-140089823955120"]
Jul 09 13:05:30.981541 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:30.981451    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg4sb\" (UniqueName: \"kubernetes.io/projected/18d89028-2156-4bb4-9a4a-70a83819cef7-kube-api-access-wg4sb\") pod \"18d89028-2156-4bb4-9a4a-70a83819cef7\" (UID: \"18d89028-2156-4bb4-9a4a-70a83819cef7\") "
Jul 09 13:05:30.983539 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:30.983511    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d89028-2156-4bb4-9a4a-70a83819cef7-kube-api-access-wg4sb" (OuterVolumeSpecName: "kube-api-access-wg4sb") pod "18d89028-2156-4bb4-9a4a-70a83819cef7" (UID: "18d89028-2156-4bb4-9a4a-70a83819cef7"). InnerVolumeSpecName "kube-api-access-wg4sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 13:05:31.082653 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:31.082616    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg4sb\" (UniqueName: \"kubernetes.io/projected/18d89028-2156-4bb4-9a4a-70a83819cef7-kube-api-access-wg4sb\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:31.721420 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:31.721377    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7471-140089823955120"
Jul 09 13:05:31.721595 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:31.721424    2574 scope.go:117] "RemoveContainer" containerID="e0f335ed0fa0250260bbfc9c7f816b6bdd653e9220111f6c8646e7e8d85f4cb1"
Jul 09 13:05:32.972308 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:32.972275    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d89028-2156-4bb4-9a4a-70a83819cef7" path="/var/lib/kubelet/pods/18d89028-2156-4bb4-9a4a-70a83819cef7/volumes"
Jul 09 13:05:43.763428 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:43.763385    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"]
Jul 09 13:05:43.764007 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:43.763670    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg" podUID="3d5e9483-7226-4fab-abf3-d305748b7052" containerName="maas-api" containerID="cri-o://e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c" gracePeriod=30
Jul 09 13:05:44.027936 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.027868    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 13:05:44.098373 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.098331    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9483-7226-4fab-abf3-d305748b7052-openshift-service-ca\") pod \"3d5e9483-7226-4fab-abf3-d305748b7052\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") "
Jul 09 13:05:44.098566 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.098478    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5e9483-7226-4fab-abf3-d305748b7052-maas-api-tls\") pod \"3d5e9483-7226-4fab-abf3-d305748b7052\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") "
Jul 09 13:05:44.098566 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.098501    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fxxx\" (UniqueName: \"kubernetes.io/projected/3d5e9483-7226-4fab-abf3-d305748b7052-kube-api-access-5fxxx\") pod \"3d5e9483-7226-4fab-abf3-d305748b7052\" (UID: \"3d5e9483-7226-4fab-abf3-d305748b7052\") "
Jul 09 13:05:44.098763 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.098732    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d5e9483-7226-4fab-abf3-d305748b7052-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d5e9483-7226-4fab-abf3-d305748b7052" (UID: "3d5e9483-7226-4fab-abf3-d305748b7052"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 13:05:44.100601 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.100568    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d5e9483-7226-4fab-abf3-d305748b7052-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d5e9483-7226-4fab-abf3-d305748b7052" (UID: "3d5e9483-7226-4fab-abf3-d305748b7052"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 13:05:44.100751 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.100662    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5e9483-7226-4fab-abf3-d305748b7052-kube-api-access-5fxxx" (OuterVolumeSpecName: "kube-api-access-5fxxx") pod "3d5e9483-7226-4fab-abf3-d305748b7052" (UID: "3d5e9483-7226-4fab-abf3-d305748b7052"). InnerVolumeSpecName "kube-api-access-5fxxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 13:05:44.199561 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.199521    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5e9483-7226-4fab-abf3-d305748b7052-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:44.199561 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.199557    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fxxx\" (UniqueName: \"kubernetes.io/projected/3d5e9483-7226-4fab-abf3-d305748b7052-kube-api-access-5fxxx\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:44.199561 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.199570    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5e9483-7226-4fab-abf3-d305748b7052-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:44.794077 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.794039    2574 generic.go:356] "Generic (PLEG): container finished" podID="3d5e9483-7226-4fab-abf3-d305748b7052" containerID="e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c" exitCode=0
Jul 09 13:05:44.794502 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.794081    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"
Jul 09 13:05:44.794502 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.794125    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg" event={"ID":"3d5e9483-7226-4fab-abf3-d305748b7052","Type":"ContainerDied","Data":"e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c"}
Jul 09 13:05:44.794502 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.794180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg" event={"ID":"3d5e9483-7226-4fab-abf3-d305748b7052","Type":"ContainerDied","Data":"c43d82d42956ed3ce606c949b016d0d6fe9d5bb2fa525d9bab6192ece384d7c0"}
Jul 09 13:05:44.794502 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.794204    2574 scope.go:117] "RemoveContainer" containerID="e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c"
Jul 09 13:05:44.804412 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.804392    2574 scope.go:117] "RemoveContainer" containerID="e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c"
Jul 09 13:05:44.804715 ip-10-0-140-237 kubenswrapper[2574]: E0709 13:05:44.804694    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c\": container with ID starting with e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c not found: ID does not exist" containerID="e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c"
Jul 09 13:05:44.804786 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.804725    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c"} err="failed to get container status \"e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c\": rpc error: code = NotFound desc = could not find container \"e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c\": container with ID starting with e25d91ccc6b4ab110923d31d1990b2619bf937b1ea69f0cbee0e0cfc0a9c9a7c not found: ID does not exist"
Jul 09 13:05:44.817163 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.817135    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"]
Jul 09 13:05:44.823330 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.823302    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-83cace-758546c569-q78sg"]
Jul 09 13:05:44.971191 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:44.971156    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5e9483-7226-4fab-abf3-d305748b7052" path="/var/lib/kubelet/pods/3d5e9483-7226-4fab-abf3-d305748b7052/volumes"
Jul 09 13:05:50.232500 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:50.232460    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"]
Jul 09 13:05:50.825626 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:50.825568    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy" containerID="cri-o://a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6" gracePeriod=30
Jul 09 13:05:51.463912 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.463872    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"]
Jul 09 13:05:51.464382 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.464145    2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" podUID="a0e6497b-95ec-4bae-af36-6ab3d0ce684a" containerName="maas-api" containerID="cri-o://1d1af852fe49b2f413e8a6d81f25994b4f7b029c9ca55f19c74427a83849db2f" gracePeriod=30
Jul 09 13:05:51.832601 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.832566    2574 generic.go:356] "Generic (PLEG): container finished" podID="a0e6497b-95ec-4bae-af36-6ab3d0ce684a" containerID="1d1af852fe49b2f413e8a6d81f25994b4f7b029c9ca55f19c74427a83849db2f" exitCode=0
Jul 09 13:05:51.832807 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.832655    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" event={"ID":"a0e6497b-95ec-4bae-af36-6ab3d0ce684a","Type":"ContainerDied","Data":"1d1af852fe49b2f413e8a6d81f25994b4f7b029c9ca55f19c74427a83849db2f"}
Jul 09 13:05:51.926478 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.926450    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 13:05:51.977285 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.977250    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls\") pod \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") "
Jul 09 13:05:51.977444 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.977314    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-openshift-service-ca\") pod \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") "
Jul 09 13:05:51.977444 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.977407    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpfpc\" (UniqueName: \"kubernetes.io/projected/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-kube-api-access-kpfpc\") pod \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\" (UID: \"a0e6497b-95ec-4bae-af36-6ab3d0ce684a\") "
Jul 09 13:05:51.977828 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.977801    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0e6497b-95ec-4bae-af36-6ab3d0ce684a" (UID: "a0e6497b-95ec-4bae-af36-6ab3d0ce684a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 13:05:51.979651 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.979606    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-kube-api-access-kpfpc" (OuterVolumeSpecName: "kube-api-access-kpfpc") pod "a0e6497b-95ec-4bae-af36-6ab3d0ce684a" (UID: "a0e6497b-95ec-4bae-af36-6ab3d0ce684a"). InnerVolumeSpecName "kube-api-access-kpfpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 13:05:51.979825 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:51.979698    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0e6497b-95ec-4bae-af36-6ab3d0ce684a" (UID: "a0e6497b-95ec-4bae-af36-6ab3d0ce684a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 13:05:52.079063 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.078910    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-maas-api-tls\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:52.079063 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.078953    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-openshift-service-ca\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:52.079063 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.078968    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpfpc\" (UniqueName: \"kubernetes.io/projected/a0e6497b-95ec-4bae-af36-6ab3d0ce684a-kube-api-access-kpfpc\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:52.840063 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.840037    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"
Jul 09 13:05:52.840478 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.840036    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2" event={"ID":"a0e6497b-95ec-4bae-af36-6ab3d0ce684a","Type":"ContainerDied","Data":"ce6d261420f52f6cc772e4e6db61cf7c147e72695c6b688017575ad376aa48c7"}
Jul 09 13:05:52.840478 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.840164    2574 scope.go:117] "RemoveContainer" containerID="1d1af852fe49b2f413e8a6d81f25994b4f7b029c9ca55f19c74427a83849db2f"
Jul 09 13:05:52.866942 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.866912    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"]
Jul 09 13:05:52.870255 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.870226    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d5cca2-7675fd987c-qb6p2"]
Jul 09 13:05:52.971950 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:52.971915    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e6497b-95ec-4bae-af36-6ab3d0ce684a" path="/var/lib/kubelet/pods/a0e6497b-95ec-4bae-af36-6ab3d0ce684a/volumes"
Jul 09 13:05:56.095154 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.095081    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 13:05:56.217805 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.217769    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-socket\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.217996 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.217836    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-credential-socket\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.217996 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.217875    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istiod-ca-cert\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.217996 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.217916    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-podinfo\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.217996 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.217957    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-data\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.218219 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218087    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-token\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.218219 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218154    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-envoy\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.218219 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218084    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 13:05:56.218219 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218150    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 13:05:56.218436 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218229    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-certs\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.218436 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218261    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7sjf\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-kube-api-access-w7sjf\") pod \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\" (UID: \"13ef76ef-595f-4724-9e06-26bc50ae9ed2\") "
Jul 09 13:05:56.218436 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218359    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 13:05:56.218583 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218503    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 13:05:56.218771 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218752    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-certs\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.218855 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218775    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-workload-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.218855 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218789    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-credential-socket\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.218855 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218805    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istiod-ca-cert\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.219000 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.218976    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-data" (OuterVolumeSpecName: "istio-data") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 13:05:56.220347 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.220319    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-token" (OuterVolumeSpecName: "istio-token") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 13:05:56.220463 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.220409    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-kube-api-access-w7sjf" (OuterVolumeSpecName: "kube-api-access-w7sjf") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "kube-api-access-w7sjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 13:05:56.220593 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.220571    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 13:05:56.220593 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.220576    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13ef76ef-595f-4724-9e06-26bc50ae9ed2" (UID: "13ef76ef-595f-4724-9e06-26bc50ae9ed2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 13:05:56.319880 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.319842    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-podinfo\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.319880 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.319871    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-data\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.319880 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.319880    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-token\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.319880 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.319888    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13ef76ef-595f-4724-9e06-26bc50ae9ed2-istio-envoy\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.320152 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.319903    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7sjf\" (UniqueName: \"kubernetes.io/projected/13ef76ef-595f-4724-9e06-26bc50ae9ed2-kube-api-access-w7sjf\") on node \"ip-10-0-140-237.ec2.internal\" DevicePath \"\""
Jul 09 13:05:56.861906 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.861863    2574 generic.go:356] "Generic (PLEG): container finished" podID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerID="a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6" exitCode=0
Jul 09 13:05:56.862085 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.861915    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"
Jul 09 13:05:56.862085 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.861937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" event={"ID":"13ef76ef-595f-4724-9e06-26bc50ae9ed2","Type":"ContainerDied","Data":"a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6"}
Jul 09 13:05:56.862085 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.861991    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw" event={"ID":"13ef76ef-595f-4724-9e06-26bc50ae9ed2","Type":"ContainerDied","Data":"8863f6410cd780cc6d0d67beb44e7d8cd3e3350a7c2791652046ef1b0986352d"}
Jul 09 13:05:56.862085 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.862017    2574 scope.go:117] "RemoveContainer" containerID="a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6"
Jul 09 13:05:56.874238 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.874215    2574 scope.go:117] "RemoveContainer" containerID="ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"
Jul 09 13:05:56.883528 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.883508    2574 scope.go:117] "RemoveContainer" containerID="a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6"
Jul 09 13:05:56.883853 ip-10-0-140-237 kubenswrapper[2574]: E0709 13:05:56.883834    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6\": container with ID starting with a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6 not found: ID does not exist" containerID="a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6"
Jul 09 13:05:56.883908 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.883865    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6"} err="failed to get container status \"a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6\": rpc error: code = NotFound desc = could not find container \"a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6\": container with ID starting with a5b9391fba6fd1258bd5c1e642a0e7f7ba5ccf24ef09ef4a90f64d007789c5f6 not found: ID does not exist"
Jul 09 13:05:56.883908 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.883882    2574 scope.go:117] "RemoveContainer" containerID="ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"
Jul 09 13:05:56.884150 ip-10-0-140-237 kubenswrapper[2574]: E0709 13:05:56.884130    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799\": container with ID starting with ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799 not found: ID does not exist" containerID="ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"
Jul 09 13:05:56.884187 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.884157    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799"} err="failed to get container status \"ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799\": rpc error: code = NotFound desc = could not find container \"ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799\": container with ID starting with ea4cf751913138462268252c335e9b5bc701d02d774d66cf9a0ab448566ab799 not found: ID does not exist"
Jul 09 13:05:56.889336 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.889309    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"]
Jul 09 13:05:56.894110 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.894084    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-83cace-openshift-default-d8b46cbcf-tvtxw"]
Jul 09 13:05:56.971883 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:56.971848    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" path="/var/lib/kubelet/pods/13ef76ef-595f-4724-9e06-26bc50ae9ed2/volumes"
Jul 09 13:05:58.248904 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:05:58.248866    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c88d8b957-f6f5m_53ed4081-acd3-4d47-9783-d807e86b36b7/authorino/0.log"
Jul 09 13:06:03.397597 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:03.397563    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hrwmn_a979de88-4fdf-45a7-86c3-8fc900949cd6/manager/0.log"
Jul 09 13:06:03.522852 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:03.522767    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fzbfg_cbccf436-b3e3-4ea7-b8be-0108bfe60a82/manager/0.log"
Jul 09 13:06:03.778971 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:03.778887    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8m6g5_2af48309-8c34-4793-81b6-161b6bc05543/server/0.log"
Jul 09 13:06:03.903091 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:03.903060    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gpsh7_90d9cc88-72fc-4dd3-969a-4680b2bafc44/manager/0.log"
Jul 09 13:06:04.296363 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:04.296332    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578cb9c586-vhgg5_e655918b-ac7a-4289-8251-15a6a68078ce/copy-manifests/0.log"
Jul 09 13:06:04.303317 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:04.303285    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578cb9c586-vhgg5_e655918b-ac7a-4289-8251-15a6a68078ce/manager/0.log"
Jul 09 13:06:04.665234 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:04.665151    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74cf5c9f5d-fgccq_47a77dd0-b266-4244-8a84-5d4e6bba2f4b/maas-api/0.log"
Jul 09 13:06:04.783874 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:04.783841    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ds52m_fe0bb06b-018c-4554-b6b8-056a545b72d6/postgres/0.log"
Jul 09 13:06:06.128459 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:06.128425    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c88d8b957-f6f5m_53ed4081-acd3-4d47-9783-d807e86b36b7/authorino/0.log"
Jul 09 13:06:06.245535 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:06.245509    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-stw7q_6691dac7-b236-45c3-a63b-8adbbd08266b/manager/0.log"
Jul 09 13:06:06.363114 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:06.363081    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5c6wp_f07563b9-bc90-4fe9-a2cd-3dec3a121347/manager/0.log"
Jul 09 13:06:06.600804 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:06.600773    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b7l49_cda6b1a9-2f83-41ae-92eb-1d7538c47b3e/registry-server/0.log"
Jul 09 13:06:06.984430 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:06.984397    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dltzc_5cc855e4-d845-4bd1-9098-b653405552cf/manager/0.log"
Jul 09 13:06:07.356037 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:07.355950    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sbf5z_3d473c20-cdc7-45e7-9fb8-b241ae844d28/istio-proxy/0.log"
Jul 09 13:06:07.741310 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:07.741283    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7776979d9-blj2f_ea98560d-f9c3-4522-b760-ddc91625dcf9/kube-auth-proxy/0.log"
Jul 09 13:06:07.865583 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:07.865549    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/1.log"
Jul 09 13:06:08.629135 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:08.629087    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tzkh2_b6f7b374-00ee-416d-bf8e-82e38c3b198e/main/0.log"
Jul 09 13:06:08.863796 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:08.863758    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-t7mjt_1758a160-9fc0-4bf5-9a46-815a133db5bb/main/0.log"
Jul 09 13:06:13.768653 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.768617    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wh67t/must-gather-lxkjl"]
Jul 09 13:06:13.769079 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769063    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy"
Jul 09 13:06:13.769128 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769081    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy"
Jul 09 13:06:13.769128 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769103    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy"
Jul 09 13:06:13.769128 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769112    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy"
Jul 09 13:06:13.769128 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769121    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5e9483-7226-4fab-abf3-d305748b7052" containerName="maas-api"
Jul 09 13:06:13.769128 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769128    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5e9483-7226-4fab-abf3-d305748b7052" containerName="maas-api"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769135    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e6497b-95ec-4bae-af36-6ab3d0ce684a" containerName="maas-api"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769140    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e6497b-95ec-4bae-af36-6ab3d0ce684a" containerName="maas-api"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769152    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d89028-2156-4bb4-9a4a-70a83819cef7" containerName="test-curl-7471-140089823955120"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769158    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d89028-2156-4bb4-9a4a-70a83819cef7" containerName="test-curl-7471-140089823955120"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769228    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769235    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d89028-2156-4bb4-9a4a-70a83819cef7" containerName="test-curl-7471-140089823955120"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769243    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5e9483-7226-4fab-abf3-d305748b7052" containerName="maas-api"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769249    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e6497b-95ec-4bae-af36-6ab3d0ce684a" containerName="maas-api"
Jul 09 13:06:13.769292 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.769257    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ef76ef-595f-4724-9e06-26bc50ae9ed2" containerName="istio-proxy"
Jul 09 13:06:13.772598 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.772577    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:13.775043 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.775021    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wh67t\"/\"default-dockercfg-kf9n4\""
Jul 09 13:06:13.775043 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.775034    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wh67t\"/\"openshift-service-ca.crt\""
Jul 09 13:06:13.775212 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.775096    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wh67t\"/\"kube-root-ca.crt\""
Jul 09 13:06:13.778287 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.778261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/219f2c0d-b953-4064-abb7-9194abdb0ea7-must-gather-output\") pod \"must-gather-lxkjl\" (UID: \"219f2c0d-b953-4064-abb7-9194abdb0ea7\") " pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:13.778413 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.778311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87hfp\" (UniqueName: \"kubernetes.io/projected/219f2c0d-b953-4064-abb7-9194abdb0ea7-kube-api-access-87hfp\") pod \"must-gather-lxkjl\" (UID: \"219f2c0d-b953-4064-abb7-9194abdb0ea7\") " pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:13.783407 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.783383    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wh67t/must-gather-lxkjl"]
Jul 09 13:06:13.878869 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.878822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/219f2c0d-b953-4064-abb7-9194abdb0ea7-must-gather-output\") pod \"must-gather-lxkjl\" (UID: \"219f2c0d-b953-4064-abb7-9194abdb0ea7\") " pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:13.879067 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.878881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87hfp\" (UniqueName: \"kubernetes.io/projected/219f2c0d-b953-4064-abb7-9194abdb0ea7-kube-api-access-87hfp\") pod \"must-gather-lxkjl\" (UID: \"219f2c0d-b953-4064-abb7-9194abdb0ea7\") " pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:13.879256 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.879231    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/219f2c0d-b953-4064-abb7-9194abdb0ea7-must-gather-output\") pod \"must-gather-lxkjl\" (UID: \"219f2c0d-b953-4064-abb7-9194abdb0ea7\") " pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:13.886694 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:13.886657    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87hfp\" (UniqueName: \"kubernetes.io/projected/219f2c0d-b953-4064-abb7-9194abdb0ea7-kube-api-access-87hfp\") pod \"must-gather-lxkjl\" (UID: \"219f2c0d-b953-4064-abb7-9194abdb0ea7\") " pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:14.082426 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:14.082338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wh67t/must-gather-lxkjl"
Jul 09 13:06:14.228944 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:14.228912    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wh67t/must-gather-lxkjl"]
Jul 09 13:06:14.231739 ip-10-0-140-237 kubenswrapper[2574]: W0709 13:06:14.231708    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod219f2c0d_b953_4064_abb7_9194abdb0ea7.slice/crio-292f26850385ea0e3c0d2ea51d133b10fbdb7e964cd537125478cf743fe64aa3 WatchSource:0}: Error finding container 292f26850385ea0e3c0d2ea51d133b10fbdb7e964cd537125478cf743fe64aa3: Status 404 returned error can't find the container with id 292f26850385ea0e3c0d2ea51d133b10fbdb7e964cd537125478cf743fe64aa3
Jul 09 13:06:14.956002 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:14.955964    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh67t/must-gather-lxkjl" event={"ID":"219f2c0d-b953-4064-abb7-9194abdb0ea7","Type":"ContainerStarted","Data":"292f26850385ea0e3c0d2ea51d133b10fbdb7e964cd537125478cf743fe64aa3"}
Jul 09 13:06:15.963557 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:15.963518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh67t/must-gather-lxkjl" event={"ID":"219f2c0d-b953-4064-abb7-9194abdb0ea7","Type":"ContainerStarted","Data":"cb386c0e2b422c8f7ee86278b124cfa352de9d41df2b9272d1ac9113a5bc6f11"}
Jul 09 13:06:15.963557 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:15.963564    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh67t/must-gather-lxkjl" event={"ID":"219f2c0d-b953-4064-abb7-9194abdb0ea7","Type":"ContainerStarted","Data":"7adf3cf39a091018aed57ac09ba93c6a2fe18626a3aab61924be9df7d443334e"}
Jul 09 13:06:15.980625 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:15.980563    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wh67t/must-gather-lxkjl" podStartSLOduration=2.114629278 podStartE2EDuration="2.980545984s" totalImagesPullingTime="865.916706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:06:15.978571053 +0000 UTC m=+2533.573195122" watchObservedRunningTime="2026-07-09 13:06:15.980545984 +0000 UTC m=+2533.575170027"
Jul 09 13:06:16.763427 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:16.763391    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcrp2_4f60c826-cf0e-4fe6-bb51-449024556554/global-pull-secret-syncer/0.log"
Jul 09 13:06:16.928187 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:16.928159    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w7vr7_26cadada-4368-4c23-a200-d7f591eb9ce0/konnectivity-agent/0.log"
Jul 09 13:06:16.949629 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:16.949595    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-237.ec2.internal_4828e88c979d58690bb239448cc9682f/haproxy/0.log"
Jul 09 13:06:22.542232 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:22.542180    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c88d8b957-f6f5m_53ed4081-acd3-4d47-9783-d807e86b36b7/authorino/0.log"
Jul 09 13:06:22.574991 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:22.574957    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-stw7q_6691dac7-b236-45c3-a63b-8adbbd08266b/manager/0.log"
Jul 09 13:06:22.609241 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:22.609153    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5c6wp_f07563b9-bc90-4fe9-a2cd-3dec3a121347/manager/0.log"
Jul 09 13:06:22.696090 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:22.696049    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b7l49_cda6b1a9-2f83-41ae-92eb-1d7538c47b3e/registry-server/0.log"
Jul 09 13:06:22.913718 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:22.912549    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dltzc_5cc855e4-d845-4bd1-9098-b653405552cf/manager/0.log"
Jul 09 13:06:25.171541 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:25.171509    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkxnv_9e97cb6e-3ac8-4e10-a678-54706377d33e/node-exporter/0.log"
Jul 09 13:06:25.196477 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:25.196435    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkxnv_9e97cb6e-3ac8-4e10-a678-54706377d33e/kube-rbac-proxy/0.log"
Jul 09 13:06:25.219491 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:25.219457    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkxnv_9e97cb6e-3ac8-4e10-a678-54706377d33e/init-textfile/0.log"
Jul 09 13:06:25.633853 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:25.633815    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-h4zzl_f3729668-4a62-4863-9f85-fe9ce337689b/prometheus-operator-admission-webhook/0.log"
Jul 09 13:06:27.049121 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.049086    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-v57sx_0e7f1626-e51b-4256-b4d9-86c4097246b6/networking-console-plugin/0.log"
Jul 09 13:06:27.175519 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.175483    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"]
Jul 09 13:06:27.183377 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.183345    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.192919 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.192887    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"]
Jul 09 13:06:27.231327 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.231280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sz2g\" (UniqueName: \"kubernetes.io/projected/4530bf57-8459-4570-842e-a782af3eec31-kube-api-access-9sz2g\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.232035 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.232001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-podres\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.232212 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.232089    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-lib-modules\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.232648 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.232631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-proc\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.232905 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.232891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-sys\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334443 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334342    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-proc\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334443 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-sys\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334443 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334434    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sz2g\" (UniqueName: \"kubernetes.io/projected/4530bf57-8459-4570-842e-a782af3eec31-kube-api-access-9sz2g\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334764 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-podres\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334764 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334504    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-sys\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334764 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-lib-modules\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334764 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334502    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-proc\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334764 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334638    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-podres\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.334764 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.334747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4530bf57-8459-4570-842e-a782af3eec31-lib-modules\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.343997 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.343964    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sz2g\" (UniqueName: \"kubernetes.io/projected/4530bf57-8459-4570-842e-a782af3eec31-kube-api-access-9sz2g\") pod \"perf-node-gather-daemonset-v4p89\" (UID: \"4530bf57-8459-4570-842e-a782af3eec31\") " pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.499869 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.499833    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:27.701467 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.700912    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"]
Jul 09 13:06:27.706843 ip-10-0-140-237 kubenswrapper[2574]: W0709 13:06:27.706802    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4530bf57_8459_4570_842e_a782af3eec31.slice/crio-6e4ba52f2276fb445462abcd7377b5ca205a60dc5984fa16e3bfacd1750d65c8 WatchSource:0}: Error finding container 6e4ba52f2276fb445462abcd7377b5ca205a60dc5984fa16e3bfacd1750d65c8: Status 404 returned error can't find the container with id 6e4ba52f2276fb445462abcd7377b5ca205a60dc5984fa16e3bfacd1750d65c8
Jul 09 13:06:27.746902 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.746876    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/2.log"
Jul 09 13:06:27.752742 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:27.752711    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vvrdl_cbc04df1-3e47-4c73-b3c8-54bd9aab3ab4/console-operator/3.log"
Jul 09 13:06:28.056241 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:28.056205    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89" event={"ID":"4530bf57-8459-4570-842e-a782af3eec31","Type":"ContainerStarted","Data":"2ff8757a8b085e75c5243409236c368762c72e4efd5c3385c9f4a243d89b7a8d"}
Jul 09 13:06:28.056632 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:28.056260    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89" event={"ID":"4530bf57-8459-4570-842e-a782af3eec31","Type":"ContainerStarted","Data":"6e4ba52f2276fb445462abcd7377b5ca205a60dc5984fa16e3bfacd1750d65c8"}
Jul 09 13:06:28.056632 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:28.056326    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:28.074898 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:28.074839    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89" podStartSLOduration=1.074821899 podStartE2EDuration="1.074821899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:06:28.072642808 +0000 UTC m=+2545.667266850" watchObservedRunningTime="2026-07-09 13:06:28.074821899 +0000 UTC m=+2545.669445943"
Jul 09 13:06:28.232818 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:28.232788    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59cd66ddfc-hmcnv_19e5a5bd-f739-4634-abbc-8f57ab488531/console/0.log"
Jul 09 13:06:28.270989 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:28.270939    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-fgmgf_92c091f5-c366-46fe-9012-9a05a80a2898/download-server/0.log"
Jul 09 13:06:30.026390 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:30.026362    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xf2k_c060fbb9-db12-482e-8f9c-7fce74c25e48/dns/0.log"
Jul 09 13:06:30.052975 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:30.052929    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xf2k_c060fbb9-db12-482e-8f9c-7fce74c25e48/kube-rbac-proxy/0.log"
Jul 09 13:06:30.194009 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:30.193977    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5fnq2_60223d89-bcc2-4415-9dd6-c3bd15f1b115/dns-node-resolver/0.log"
Jul 09 13:06:30.897231 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:30.897203    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xbjjh_a7f8befd-7746-416c-94a1-c78e5260a13a/node-ca/0.log"
Jul 09 13:06:31.893240 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:31.893211    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sbf5z_3d473c20-cdc7-45e7-9fb8-b241ae844d28/istio-proxy/0.log"
Jul 09 13:06:32.080390 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:32.080359    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7776979d9-blj2f_ea98560d-f9c3-4522-b760-ddc91625dcf9/kube-auth-proxy/0.log"
Jul 09 13:06:32.114859 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:32.114831    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/0.log"
Jul 09 13:06:32.155467 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:32.155381    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/1.log"
Jul 09 13:06:33.005637 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:33.005601    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-scrxj_53c9a2e7-219c-4889-8e51-e467dcef0381/serve-healthcheck-canary/0.log"
Jul 09 13:06:33.545335 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:33.545304    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6t9p2_16f7ba62-d0ff-4b6c-9281-b389135d478b/kube-rbac-proxy/0.log"
Jul 09 13:06:33.568500 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:33.568471    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6t9p2_16f7ba62-d0ff-4b6c-9281-b389135d478b/exporter/0.log"
Jul 09 13:06:33.592469 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:33.592400    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6t9p2_16f7ba62-d0ff-4b6c-9281-b389135d478b/extractor/0.log"
Jul 09 13:06:34.076215 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:34.076182    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wh67t/perf-node-gather-daemonset-v4p89"
Jul 09 13:06:36.070537 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:36.070478    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hrwmn_a979de88-4fdf-45a7-86c3-8fc900949cd6/manager/0.log"
Jul 09 13:06:36.261335 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:36.261265    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fzbfg_cbccf436-b3e3-4ea7-b8be-0108bfe60a82/manager/0.log"
Jul 09 13:06:36.396180 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:36.396110    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8m6g5_2af48309-8c34-4793-81b6-161b6bc05543/server/0.log"
Jul 09 13:06:36.511314 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:36.511217    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gpsh7_90d9cc88-72fc-4dd3-969a-4680b2bafc44/manager/0.log"
Jul 09 13:06:36.746138 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:36.746103    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578cb9c586-vhgg5_e655918b-ac7a-4289-8251-15a6a68078ce/manager/0.log"
Jul 09 13:06:36.791263 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:36.791226    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578cb9c586-vhgg5_e655918b-ac7a-4289-8251-15a6a68078ce/copy-manifests/0.log"
Jul 09 13:06:37.442185 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:37.442155    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sbf5z_3d473c20-cdc7-45e7-9fb8-b241ae844d28/istio-proxy/0.log"
Jul 09 13:06:37.579005 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:37.578930    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7776979d9-blj2f_ea98560d-f9c3-4522-b760-ddc91625dcf9/kube-auth-proxy/0.log"
Jul 09 13:06:37.602698 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:37.602648    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/0.log"
Jul 09 13:06:37.637756 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:37.637726    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/1.log"
Jul 09 13:06:38.154249 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:38.154192    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jzfrq_d9afa0d2-94fa-47be-81f6-3ff93f5ff182/openshift-lws-operator/0.log"
Jul 09 13:06:38.965752 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:38.965720    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sbf5z_3d473c20-cdc7-45e7-9fb8-b241ae844d28/istio-proxy/0.log"
Jul 09 13:06:39.108492 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:39.108412    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7776979d9-blj2f_ea98560d-f9c3-4522-b760-ddc91625dcf9/kube-auth-proxy/0.log"
Jul 09 13:06:39.136639 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:39.136598    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/0.log"
Jul 09 13:06:39.179427 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:39.179390    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fw8tm_934a66ff-914a-409a-911a-b8e31e3661f1/istio-proxy/1.log"
Jul 09 13:06:45.007511 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.007477    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/kube-multus-additional-cni-plugins/0.log"
Jul 09 13:06:45.039355 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.039327    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/egress-router-binary-copy/0.log"
Jul 09 13:06:45.067611 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.067572    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/cni-plugins/0.log"
Jul 09 13:06:45.092820 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.092795    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/bond-cni-plugin/0.log"
Jul 09 13:06:45.119766 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.119736    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/routeoverride-cni/0.log"
Jul 09 13:06:45.145966 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.145920    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/whereabouts-cni-bincopy/0.log"
Jul 09 13:06:45.168911 ip-10-0-140-237 kubenswrapper[2574]: I0709 13:06:45.168882    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lqp8_5aa3ac72-32b5-4188-a6ca-72772cad5b90/whereabouts-cni/0.log"