Jul 21 16:59:57.797138 ip-10-0-140-24 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 16:59:57.797155 ip-10-0-140-24 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 16:59:57.797165 ip-10-0-140-24 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 16:59:57.797633 ip-10-0-140-24 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 17:00:07.814487 ip-10-0-140-24 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 17:00:07.814502 ip-10-0-140-24 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6bd3c9094ddc470196501090a70039ca --
Jul 21 17:01:59.915864 ip-10-0-140-24 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 17:02:00.321700 ip-10-0-140-24 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 17:02:00.321700 ip-10-0-140-24 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 17:02:00.321700 ip-10-0-140-24 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 17:02:00.321700 ip-10-0-140-24 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 17:02:00.321700 ip-10-0-140-24 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 17:02:00.323737 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.323573    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 17:02:00.327738 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327723    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 17:02:00.327738 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327738    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327743    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327748    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327751    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327754    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327757    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327760    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327763    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327766    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327769    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327771    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327774    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327777    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327781    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327784    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327787    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327790    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327793    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327795    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327798    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 17:02:00.327810 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327800    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327803    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327805    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327808    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327812    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327816    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327820    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327823    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327827    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327831    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327834    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327837    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327840    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327842    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327845    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327847    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327850    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327853    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327856    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 17:02:00.328291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327859    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327861    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327864    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327866    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327868    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327871    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327874    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327877    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327879    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327882    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327884    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327886    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327889    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327891    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327903    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327907    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327910    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327913    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327917    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 17:02:00.328777 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327919    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327922    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327925    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327927    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327930    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327933    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327936    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327939    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327942    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327945    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327948    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327950    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327953    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327956    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327959    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327961    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327964    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327966    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327970    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327973    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 17:02:00.329248 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327975    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327978    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327981    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327983    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327986    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327988    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327990    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327993    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327996    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.327999    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328001    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328004    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328007    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328009    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328012    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328014    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328017    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328019    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328022    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 17:02:00.329757 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328024    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328741    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328749    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328752    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328755    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328758    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328761    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328764    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328767    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328770    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328772    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328775    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328777    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328780    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328784    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328788    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328791    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328793    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328796    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328799    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 17:02:00.330226 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328801    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328806    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328809    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328812    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328814    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328817    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328820    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328822    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328825    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328827    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328829    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328832    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328834    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328837    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328840    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328842    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328844    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328847    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328849    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328852    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 17:02:00.330783 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328855    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328857    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328859    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328863    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328865    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328868    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328871    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328873    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328876    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328878    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328880    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328883    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328885    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328888    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328890    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328893    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328896    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328900    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328903    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 17:02:00.331269 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328905    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328909    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328912    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328916    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328918    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328921    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328923    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328926    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328928    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328931    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328933    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328936    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328938    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328944    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328947    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328949    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328952    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328955    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328957    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 17:02:00.331760 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328960    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328962    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328965    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328967    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328970    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328972    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328974    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328978    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328980    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328983    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328985    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328987    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328991    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328993    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328996    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.328998    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329001    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329003    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329005    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329008    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 17:02:00.332220 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329010    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329013    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329959    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329965    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329969    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329972    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329982    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329985    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329988    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329990    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329993    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329996    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.329999    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330001    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330004    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330006    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330009    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330011    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330014    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330016    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 17:02:00.332829 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330019    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330021    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330024    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330027    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330031    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330034    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330036    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330038    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330041    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330043    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330046    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330048    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330050    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330053    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330056    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330058    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330061    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330063    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330066    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330073    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 17:02:00.333303 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330076    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330079    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330081    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330084    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330086    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330088    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330091    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330093    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330096    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330098    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330100    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330103    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330105    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330107    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330110    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330112    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330114    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330117    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330122    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330124    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 17:02:00.333815 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330126    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330129    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330131    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330133    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330137    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330139    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330141    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330144    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330146    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330148    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330151    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330153    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330161    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330165    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330167    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330170    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330172    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330175    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330178    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330180    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 17:02:00.334311 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330182    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330185    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330187    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330190    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330193    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330196    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330198    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330201    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330205    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330208    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330212    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330215    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330218    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330220    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330223    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330225    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330228    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330231    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330235    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 17:02:00.334836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330238    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330241    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330317    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330328    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330334    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330338    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330360    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330363    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330367    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330371    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330374    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330377    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330381    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330384    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330387    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330390    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330393    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330395    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330398    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330401    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330405    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330408    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330411    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330415    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 17:02:00.335300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330418    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330422    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330425    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330428    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330431    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330434    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330437    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330441    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330443    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330446    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330450    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330453    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330456    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330459    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330462    2561 flags.go:64] FLAG: --enable-server="true"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330466    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330470    2561 flags.go:64] FLAG: --event-burst="100"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330473    2561 flags.go:64] FLAG: --event-qps="50"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330475    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330478    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330481    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330485    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330487    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330490    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330493    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 21 17:02:00.335959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330496    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330498    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330501    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330504    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330507    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330510    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330514    2561 flags.go:64] FLAG: --feature-gates=""
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330519    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330522    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330524    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330527    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330530    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330534    2561 flags.go:64] FLAG: --help="false"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330537    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330540    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330542    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330545    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330548    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330568    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330574    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330577    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330581    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330584    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 17:02:00.336658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330588    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330591    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330593    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330596    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330599    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330602    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330604    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330607    2561 flags.go:64] FLAG: --lock-file=""
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330610    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330613    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330616    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330621    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330624    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330627    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330642    2561 flags.go:64] FLAG: --logging-format="text"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330647    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330651    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330653    2561 flags.go:64] FLAG: --manifest-url=""
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330656    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330661    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330664    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330668    2561 flags.go:64] FLAG: --max-pods="110"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330671    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330673    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330676    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 17:02:00.337331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330679    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330682    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330685    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330688    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330694    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330698    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330701    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330704    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330708    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330714    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330717    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330720    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330723    2561 flags.go:64] FLAG: --port="10250"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330726    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330729    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dce653d5106da898"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330732    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330734    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330737    2561 flags.go:64] FLAG: --register-node="true"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330740    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330743    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330746    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330749    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330753    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330756    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330759    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 17:02:00.338060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330761    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330764    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330767    2561 flags.go:64] FLAG: --runonce="false"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330769    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330772    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330775    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330778    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330781    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330784    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330787    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330790    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330792    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330795    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330799    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330802    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330805    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330808    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330811    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330816    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330819    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330822    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330826    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330828    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330831    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330834    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 17:02:00.338773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330836    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330839    2561 flags.go:64] FLAG: --v="2"
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330843    2561 flags.go:64] FLAG: --version="false"
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330847    2561 flags.go:64] FLAG: --vmodule=""
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330852    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.330855    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330977    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330981    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330984    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330987    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330989    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330992    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330994    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.330998    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331000    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331003    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331005    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331008    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331010    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331013    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331016    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 17:02:00.339521 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331019    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331022    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331024    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331027    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331030    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331032    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331035    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331037    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331040    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331042    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331045    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331047    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331049    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331052    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331054    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331058    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331060    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331063    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331065    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 17:02:00.340131 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331068    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331070    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331072    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331075    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331079    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331083    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331086    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331089    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331091    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331094    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331096    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331099    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331104    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331107    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331110    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331113    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331116    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331119    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331122    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 17:02:00.340685 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331125    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331128    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331131    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331133    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331136    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331138    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331141    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331143    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331146    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331149    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331151    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331154    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331156    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331159    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331161    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331164    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331166    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331169    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331171    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331173    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 17:02:00.341175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331176    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331178    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331180    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331183    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331185    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331189    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331192    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331194    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331197    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331199    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331202    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331205    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331207    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331209    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331212    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331214    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331217    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331219    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331223    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 17:02:00.341763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331226    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331228    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331232    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331234    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331237    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331240    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.331242    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.331251    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.337939    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.337953    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.339012    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.339023    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339078    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339083    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 17:02:00.342330 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339086    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339089    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339092    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339094    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339097    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339100    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339102    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339105    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339108    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339110    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339114    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339116    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339119    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339122    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339124    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339127    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339129    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339131    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339134    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 17:02:00.342936 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339153    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339157    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339160    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339163    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339165    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339168    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339171    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339173    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339176    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339179    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339182    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339185    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339187    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339190    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339193    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339195    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339197    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339200    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339202    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339205    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 17:02:00.343711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339207    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339210    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339212    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339214    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339217    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339220    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339222    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339225    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339228    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339230    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339233    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339235    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339238    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339240    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339244    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339248    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339250    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339253    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339256    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 17:02:00.344276 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339258    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339262    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339265    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339268    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339271    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339282    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339284    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339287    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339289    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339292    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339295    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339297    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339299    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339303    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339306    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339309    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339312    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339315    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339318    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339321    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 17:02:00.344836 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339323    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339326    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339330    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339333    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339336    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339338    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339341    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339343    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339345    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339348    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339350    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339352    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339355    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339358    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339361    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339364    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339366    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339369    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 17:02:00.345353 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339371    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.339377    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339501    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339506    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339509    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339511    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339514    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339517    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339521    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339524    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339526    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339529    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339531    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339535    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339537    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 17:02:00.345801 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339540    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339542    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339545    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339547    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339550    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339567    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339571    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339573    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339576    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339579    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339582    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339584    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339587    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339590    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339592    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339595    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339597    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339600    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339602    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339605    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 17:02:00.346185 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339607    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339609    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339612    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339614    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339616    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339619    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339621    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339624    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339626    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339629    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339631    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339635    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339637    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339640    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339642    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339644    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339648    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339650    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339654    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339657    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 17:02:00.346693 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339660    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339662    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339665    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339667    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339670    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339672    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339674    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339677    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339679    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339682    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339684    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339687    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339689    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339692    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339694    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339696    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339698    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339702    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339705    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 17:02:00.347180 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339708    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339711    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339713    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339716    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339718    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339722    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339725    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339728    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339730    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339734    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339737    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339740    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339742    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339744    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339747    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339749    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339752    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339754    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339757    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 17:02:00.347713 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339759    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339762    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339764    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339766    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339769    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339771    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339773    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:00.339776    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.339780    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.339880    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.343734    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.345158    2561 server.go:1019] "Starting client certificate rotation"
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.345253    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 17:02:00.348175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.345286    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 17:02:00.348509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.347466    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 17:02:00.348596 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.348578    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 17:02:00.356548 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.356531    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 21 17:02:00.358677 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.358664    2561 log.go:25] "Validated CRI v1 image API"
Jul 21 17:02:00.359974 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.359958    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 17:02:00.365004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.364981    2561 fs.go:135] Filesystem UUIDs: map[4cad8ae2-9bb7-4c3c-a87a-48207cbd03ad:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a47ae049-a370-448d-8ba6-9417f19daf96:/dev/nvme0n1p4]
Jul 21 17:02:00.365004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.365000    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 17:02:00.369159 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.369055    2561 manager.go:216] Machine: {Timestamp:2026-07-21 17:02:00.36859377 +0000 UTC m=+0.333493365 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100239 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2955a231fa8b78ee1f71b3c0649b85 SystemUUID:ec2955a2-31fa-8b78-ee1f-71b3c0649b85 BootID:6bd3c909-4ddc-4701-9650-1090a70039ca Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cb:22:e7:7c:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cb:22:e7:7c:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:24:f4:f6:f5:27 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 17:02:00.369159 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.369151    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 17:02:00.369272 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.369224    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 17:02:00.372210 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.372183    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 17:02:00.372351 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.372212    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-24.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 17:02:00.372964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.372954    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 17:02:00.373005 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.372966    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 17:02:00.373005 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.372974    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 17:02:00.373005 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.372984    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 17:02:00.374276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.374265    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 17:02:00.379691 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.379677    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 17:02:00.379821 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.379808    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 17:02:00.380661 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.380649    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 17:02:00.380695 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.380667    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 17:02:00.380695 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.380684    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 17:02:00.380695 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.380693    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 21 17:02:00.380802 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.380702    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 17:02:00.381068 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.381053    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 17:02:00.381123 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.381071    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 17:02:00.383337 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.383314    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 17:02:00.384942 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.384923    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 17:02:00.386959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.386943    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 17:02:00.387018 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.386975    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 17:02:00.389107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389094    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389112    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389119    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389124    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389132    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389137    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389146    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389151    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 17:02:00.389155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389158    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 17:02:00.389388 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389163    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 17:02:00.389388 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389171    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 17:02:00.389388 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389177    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 17:02:00.389955 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389943    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 17:02:00.389955 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.389955    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 17:02:00.393077 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.393064    2561 server.go:1268] "Started kubelet"
Jul 21 17:02:00.393192 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.393161    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 17:02:00.393297 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.393142    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 17:02:00.393411 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.393395    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 17:02:00.393911 ip-10-0-140-24 systemd[1]: Started Kubernetes Kubelet.
Jul 21 17:02:00.395349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.395336    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 17:02:00.395400 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.395352    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 17:02:00.395728 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.395712    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-24.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 17:02:00.395772 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.395729    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 17:02:00.395901 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.395882    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 17:02:00.399420 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.399404    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 17:02:00.401319 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.401303    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 17:02:00.401425 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.401323    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 17:02:00.401425 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.401410    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.401945 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.401915    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 17:02:00.401945 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.401917    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 17:02:00.401945 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.401943    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 17:02:00.402120 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402043    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 17:02:00.402120 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402052    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 17:02:00.402214 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402130    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 17:02:00.402214 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402152    2561 factory.go:55] Registering systemd factory
Jul 21 17:02:00.402214 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402163    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 21 17:02:00.402214 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.400956    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-24.ec2.internal.18c45cbd493f9792  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-24.ec2.internal,UID:ip-10-0-140-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-24.ec2.internal,},FirstTimestamp:2026-07-21 17:02:00.393037714 +0000 UTC m=+0.357937308,LastTimestamp:2026-07-21 17:02:00.393037714 +0000 UTC m=+0.357937308,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-24.ec2.internal,}"
Jul 21 17:02:00.402401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402367    2561 factory.go:153] Registering CRI-O factory
Jul 21 17:02:00.402401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402383    2561 factory.go:223] Registration of the crio container factory successfully
Jul 21 17:02:00.402401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402398    2561 factory.go:103] Registering Raw factory
Jul 21 17:02:00.402535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402407    2561 manager.go:1196] Started watching for new ooms in manager
Jul 21 17:02:00.402937 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.402924    2561 manager.go:349] Starting recovery of all containers
Jul 21 17:02:00.406666 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.406639    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 17:02:00.407074 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.407049    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 17:02:00.410126 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.410093    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 17:02:00.413211 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.413194    2561 manager.go:354] Recovery completed
Jul 21 17:02:00.419520 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.419419    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.422280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422262    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.422333 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422302    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.422333 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422318    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.422868 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422855    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 17:02:00.422868 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422863    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.422868 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422867    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 17:02:00.423014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.422885    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 17:02:00.423727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.423705    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.423767 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.423734    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.423767 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.423749    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.424317 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.424253    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-24.ec2.internal.18c45cbd4afdd8c3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-24.ec2.internal,UID:ip-10-0-140-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-24.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-24.ec2.internal,},FirstTimestamp:2026-07-21 17:02:00.422283459 +0000 UTC m=+0.387183060,LastTimestamp:2026-07-21 17:02:00.422283459 +0000 UTC m=+0.387183060,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-24.ec2.internal,}"
Jul 21 17:02:00.425070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.425056    2561 policy_none.go:49] "None policy: Start"
Jul 21 17:02:00.425146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.425075    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 17:02:00.425146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.425088    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 17:02:00.427156 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.427144    2561 policy_none.go:47] "Start"
Jul 21 17:02:00.438534 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.438473    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-24.ec2.internal.18c45cbd4afe3e7c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-24.ec2.internal,UID:ip-10-0-140-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-24.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-24.ec2.internal,},FirstTimestamp:2026-07-21 17:02:00.4223095 +0000 UTC m=+0.387209099,LastTimestamp:2026-07-21 17:02:00.4223095 +0000 UTC m=+0.387209099,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-24.ec2.internal,}"
Jul 21 17:02:00.449891 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.449822    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-24.ec2.internal.18c45cbd4afe73d9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-24.ec2.internal,UID:ip-10-0-140-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-24.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-24.ec2.internal,},FirstTimestamp:2026-07-21 17:02:00.422323161 +0000 UTC m=+0.387222757,LastTimestamp:2026-07-21 17:02:00.422323161 +0000 UTC m=+0.387222757,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-24.ec2.internal,}"
Jul 21 17:02:00.454778 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.454763    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z4sfn"
Jul 21 17:02:00.462424 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.462366    2561 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-24.ec2.internal.18c45cbd4afdd8c3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-24.ec2.internal.18c45cbd4afdd8c3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-24.ec2.internal,UID:ip-10-0-140-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-24.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-24.ec2.internal,},FirstTimestamp:2026-07-21 17:02:00.422283459 +0000 UTC m=+0.387183060,LastTimestamp:2026-07-21 17:02:00.423716745 +0000 UTC m=+0.388616342,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-24.ec2.internal,}"
Jul 21 17:02:00.465138 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.465123    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z4sfn"
Jul 21 17:02:00.471119 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471104    2561 manager.go:337] "Starting Device Plugin manager"
Jul 21 17:02:00.471199 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.471138    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 17:02:00.471199 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471151    2561 server.go:87] "Starting device plugin registration server"
Jul 21 17:02:00.471412 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471395    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 17:02:00.471503 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471412    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 17:02:00.471503 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471496    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 17:02:00.471636 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471589    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 17:02:00.471636 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.471597    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 17:02:00.472146 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.472127    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 17:02:00.472216 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.472174    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.506773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.506755    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 17:02:00.514160 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.506793    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 17:02:00.514160 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.506813    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 17:02:00.514160 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.506842    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 17:02:00.514160 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.509716    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 17:02:00.572083 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.572029    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.572862 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.572846    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.572922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.572876    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.572922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.572887    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.572922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.572914    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.595624 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.595605    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.595675 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.595635    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-24.ec2.internal\": node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.606986 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.606967    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"]
Jul 21 17:02:00.607037 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.607023    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.607820 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.607805    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.607901 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.607832    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.607901 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.607841    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.610179 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610168    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.610315 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610301    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.610350 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610331    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.610913 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610897    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.610959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610918    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.610959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610929    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.610959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610937    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.610959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610941    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.610959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.610947    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.613181 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.613163    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.613253 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.613196    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:00.613916 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.613898    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:00.613989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.613922    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:00.613989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.613931    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:00.638354 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.638338    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.639152 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.639139    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-24.ec2.internal\" not found" node="ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.643755 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.643740    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-24.ec2.internal\" not found" node="ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.704304 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.704274    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e7b7687333f4ca94a71843379ec67a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal\" (UID: \"3e7b7687333f4ca94a71843379ec67a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.704404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.704313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7b7687333f4ca94a71843379ec67a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal\" (UID: \"3e7b7687333f4ca94a71843379ec67a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.704404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.704342    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/de9305d86924dd928dcf28542838bdc5-config\") pod \"kube-apiserver-proxy-ip-10-0-140-24.ec2.internal\" (UID: \"de9305d86924dd928dcf28542838bdc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.739113 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.739085    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.805251 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.805217    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e7b7687333f4ca94a71843379ec67a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal\" (UID: \"3e7b7687333f4ca94a71843379ec67a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.805403 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.805265    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7b7687333f4ca94a71843379ec67a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal\" (UID: \"3e7b7687333f4ca94a71843379ec67a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.805403 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.805283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/de9305d86924dd928dcf28542838bdc5-config\") pod \"kube-apiserver-proxy-ip-10-0-140-24.ec2.internal\" (UID: \"de9305d86924dd928dcf28542838bdc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.805403 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.805225    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e7b7687333f4ca94a71843379ec67a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal\" (UID: \"3e7b7687333f4ca94a71843379ec67a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.805403 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.805331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/de9305d86924dd928dcf28542838bdc5-config\") pod \"kube-apiserver-proxy-ip-10-0-140-24.ec2.internal\" (UID: \"de9305d86924dd928dcf28542838bdc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.805403 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.805354    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7b7687333f4ca94a71843379ec67a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal\" (UID: \"3e7b7687333f4ca94a71843379ec67a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.840052 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.839985    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.940777 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:00.940749    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:00.942931 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.942912    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:00.946706 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:00.946674    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:01.041573 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:01.041526    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:01.142118 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:01.142045    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:01.242542 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:01.242515    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:01.344706 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.344666    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 17:02:01.344706 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:01.344697    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:01.345083 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.344869    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 17:02:01.446438 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.446380    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 17:02:01.446512 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:01.446392    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:01.447408 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.447381    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 17:02:01.447543 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.447416    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 17:02:01.447543 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.447431    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeHasSufficientPID"
Jul 21 17:02:01.470807 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.470759    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 16:57:01 +0000 UTC" deadline="2027-12-18 22:28:13.639899199 +0000 UTC"
Jul 21 17:02:01.470807 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.470797    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12365h26m12.169104964s"
Jul 21 17:02:01.529152 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.529129    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 17:02:01.546792 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:01.546765    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-24.ec2.internal\" not found"
Jul 21 17:02:01.581246 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.581226    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 17:02:01.602040 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.602011    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:01.620750 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.620728    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 17:02:01.621959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.621943    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"
Jul 21 17:02:01.638202 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.638180    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 17:02:01.668666 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:01.668643    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 17:02:02.214351 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.214322    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 17:02:02.339320 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.339273    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7b7687333f4ca94a71843379ec67a5.slice/crio-e3080ae47b36abfb304d0dc6e58e5d017873cc133ca17037ef28ac036cfe5c8c WatchSource:0}: Error finding container e3080ae47b36abfb304d0dc6e58e5d017873cc133ca17037ef28ac036cfe5c8c: Status 404 returned error can't find the container with id e3080ae47b36abfb304d0dc6e58e5d017873cc133ca17037ef28ac036cfe5c8c
Jul 21 17:02:02.347546 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.347523    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:02:02.353781 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.353759    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9305d86924dd928dcf28542838bdc5.slice/crio-957bb8e5746743522c42ee7ea481387c2df108da0e58929e3320b64c2659e06c WatchSource:0}: Error finding container 957bb8e5746743522c42ee7ea481387c2df108da0e58929e3320b64c2659e06c: Status 404 returned error can't find the container with id 957bb8e5746743522c42ee7ea481387c2df108da0e58929e3320b64c2659e06c
Jul 21 17:02:02.384011 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.383986    2561 apiserver.go:52] "Watching apiserver"
Jul 21 17:02:02.392414 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.392394    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 17:02:02.392831 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.392809    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-wnlzz","openshift-image-registry/node-ca-7k5q8","openshift-network-operator/iptables-alerter-rpn7m","openshift-ovn-kubernetes/ovnkube-node-qczpg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6","openshift-dns/node-resolver-kns2f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal","openshift-multus/multus-additional-cni-plugins-m6n8j","openshift-multus/multus-dp8bc","openshift-multus/network-metrics-daemon-wpvhl","openshift-network-diagnostics/network-check-target-d6h64","kube-system/global-pull-secret-syncer-jjmrd","kube-system/konnectivity-agent-r9ckt","kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal"]
Jul 21 17:02:02.395660 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.395640    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:02.395759 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.395736    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:02.398300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.398278    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.400640 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.400614    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.401610 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.401593    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 17:02:02.403064 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.403038    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.405471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.405454    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.407923 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.407905    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.410640 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.410602    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.411878 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.411860    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.412103 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.412088    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.412187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.412096    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zgll9\""
Jul 21 17:02:02.412669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.412651    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 17:02:02.412804 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.412694    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 17:02:02.412804 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.412759    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 17:02:02.412994 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.412978    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.413112 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.413039    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.413177 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.413149    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.413322 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.413298    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qlmtx\""
Jul 21 17:02:02.413399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.413379    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4f6p8\""
Jul 21 17:02:02.415077 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415059    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 17:02:02.415174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415093    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.415235 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415219    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-run-netns\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415294 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea544a8b-c838-491a-86aa-bc0c301fa723-iptables-alerter-script\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.415294 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415260    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-slash\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415294 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415277    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-etc-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415296    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdwb2\" (UniqueName: \"kubernetes.io/projected/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-kube-api-access-kdwb2\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415311    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd8e5c13-07a9-49bd-bf9f-08de8be07870-agent-certs\") pod \"konnectivity-agent-r9ckt\" (UID: \"dd8e5c13-07a9-49bd-bf9f-08de8be07870\") " pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415349    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-kubelet\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-systemd\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415421    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-node-log\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415449    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-cni-netd\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415475    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxfjd\" (UniqueName: \"kubernetes.io/projected/ea544a8b-c838-491a-86aa-bc0c301fa723-kube-api-access-vxfjd\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.415544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415503    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovnkube-config\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415552    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-env-overrides\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovn-node-metrics-cert\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415625    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415634    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415634    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea544a8b-c838-491a-86aa-bc0c301fa723-host-slash\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415772    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd8e5c13-07a9-49bd-bf9f-08de8be07870-konnectivity-ca\") pod \"konnectivity-agent-r9ckt\" (UID: \"dd8e5c13-07a9-49bd-bf9f-08de8be07870\") " pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415857    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovnkube-script-lib\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-systemd-units\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415941    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-var-lib-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.415979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.415989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54h9x\" (UniqueName: \"kubernetes.io/projected/4cb01e07-87f3-42a4-a130-4110a6245b5e-kube-api-access-54h9x\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.416641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.416019    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:02.416641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.416037    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-ovn\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.416641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.416092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-log-socket\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.416641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.416121    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-cni-bin\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.416641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.416165    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.417409 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.417393    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.418228 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.418211    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.418776 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.418752    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.418890 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.418803    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 17:02:02.418890 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.418812    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.419399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.419233    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hpsbv\""
Jul 21 17:02:02.419399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.419299    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 17:02:02.419399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.419312    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 17:02:02.419399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.419338    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 17:02:02.419587 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.419481    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 17:02:02.421131 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.421107    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mn262\""
Jul 21 17:02:02.421221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.421115    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:02.421297 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.421268    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:02.425750 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.425728    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.425846 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.425798    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:02.428593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.428517    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 17:02:02.428733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.428613    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.428733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.428617    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 17:02:02.428733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.428672    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.429679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429485    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 17:02:02.429679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429490    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 17:02:02.429679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429512    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ll7nw\""
Jul 21 17:02:02.429679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429536    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 17:02:02.429679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429612    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xjmn8\""
Jul 21 17:02:02.429679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429498    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 17:02:02.429954 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429713    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mdvht\""
Jul 21 17:02:02.429954 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.429730    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5r9rg\""
Jul 21 17:02:02.444728 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.444712    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 17:02:02.497071 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.497014    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tznrf"
Jul 21 17:02:02.503251 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.503235    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 17:02:02.508878 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.508863    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tznrf"
Jul 21 17:02:02.510749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.510714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal" event={"ID":"3e7b7687333f4ca94a71843379ec67a5","Type":"ContainerStarted","Data":"e3080ae47b36abfb304d0dc6e58e5d017873cc133ca17037ef28ac036cfe5c8c"}
Jul 21 17:02:02.511795 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.511770    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal" event={"ID":"de9305d86924dd928dcf28542838bdc5","Type":"ContainerStarted","Data":"957bb8e5746743522c42ee7ea481387c2df108da0e58929e3320b64c2659e06c"}
Jul 21 17:02:02.516821 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516804    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-kubelet\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.516903 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516835    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-node-log\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.516903 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516863    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.516903 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-cnibin\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.517014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516914    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-kubernetes\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.517014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516918    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-kubelet\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516917    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-node-log\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516954    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-lib-modules\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.517014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.516990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-etc-kubernetes\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517019    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-env-overrides\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517037    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovn-node-metrics-cert\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-registration-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517076    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-sys-fs\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpb6x\" (UniqueName: \"kubernetes.io/projected/6407cbcb-cfef-4674-8527-25513af84e1e-kube-api-access-jpb6x\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-netns\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517150    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysconfig\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.517185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517176    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e4892ce1-b232-4ced-b142-289d7804ecff-hosts-file\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-k8s-cni-cncf-io\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-conf-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cni-binary-copy\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517287    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517310    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-tuned\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517335    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgkhr\" (UniqueName: \"kubernetes.io/projected/8ed095f6-a14d-4e47-8778-d650a46ade19-kube-api-access-pgkhr\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517365    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd8e5c13-07a9-49bd-bf9f-08de8be07870-konnectivity-ca\") pod \"konnectivity-agent-r9ckt\" (UID: \"dd8e5c13-07a9-49bd-bf9f-08de8be07870\") " pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517451    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovnkube-script-lib\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517453    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517481    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-socket-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517490    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-env-overrides\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517511    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.517570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517551    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517595    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c65d0345-d0cf-40c2-9bd7-0086726a8f31-serviceca\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-systemd-units\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517651    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4892ce1-b232-4ced-b142-289d7804ecff-tmp-dir\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517676    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-socket-dir-parent\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517700    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fmhr\" (UniqueName: \"kubernetes.io/projected/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-kube-api-access-9fmhr\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517731    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517735    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-systemd-units\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517763    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-ovn\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517787    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-log-socket\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517813    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-device-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517820    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-ovn\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.517829    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517839    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-system-cni-dir\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517870    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-log-socket\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.517892    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:02:03.017868992 +0000 UTC m=+2.982768576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517928    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd8e5c13-07a9-49bd-bf9f-08de8be07870-konnectivity-ca\") pod \"konnectivity-agent-r9ckt\" (UID: \"dd8e5c13-07a9-49bd-bf9f-08de8be07870\") " pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.518262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517929    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-os-release\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-run\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.517994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovnkube-script-lib\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea544a8b-c838-491a-86aa-bc0c301fa723-iptables-alerter-script\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-os-release\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kxbb\" (UniqueName: \"kubernetes.io/projected/3887f0b3-3a95-42c6-943b-d945d53e8ac9-kube-api-access-4kxbb\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518090    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psbnx\" (UniqueName: \"kubernetes.io/projected/c65d0345-d0cf-40c2-9bd7-0086726a8f31-kube-api-access-psbnx\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518120    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ed095f6-a14d-4e47-8778-d650a46ade19-tmp\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwb2\" (UniqueName: \"kubernetes.io/projected/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-kube-api-access-kdwb2\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-systemd\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518179    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd8e5c13-07a9-49bd-bf9f-08de8be07870-agent-certs\") pod \"konnectivity-agent-r9ckt\" (UID: \"dd8e5c13-07a9-49bd-bf9f-08de8be07870\") " pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518200    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-systemd\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-cni-netd\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518234    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518261    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysctl-d\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518275    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-run-systemd\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518286    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c24b7c8-f986-4410-bf06-5d31d80f43b6-kubelet-config\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.518989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518322    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c24b7c8-f986-4410-bf06-5d31d80f43b6-dbus\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518346    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxfjd\" (UniqueName: \"kubernetes.io/projected/ea544a8b-c838-491a-86aa-bc0c301fa723-kube-api-access-vxfjd\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518285    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-cni-netd\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518381    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovnkube-config\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518544    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-multus-certs\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518592    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-sys\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea544a8b-c838-491a-86aa-bc0c301fa723-host-slash\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518620    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea544a8b-c838-491a-86aa-bc0c301fa723-iptables-alerter-script\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518639    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-cni-multus\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518663    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c65d0345-d0cf-40c2-9bd7-0086726a8f31-host\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518687    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysctl-conf\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518687    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea544a8b-c838-491a-86aa-bc0c301fa723-host-slash\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518761    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-hostroot\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518806    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-var-lib-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.519705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518831    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518863    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54h9x\" (UniqueName: \"kubernetes.io/projected/4cb01e07-87f3-42a4-a130-4110a6245b5e-kube-api-access-54h9x\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518866    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-var-lib-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518888    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-system-cni-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518893    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518916    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-cni-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518940    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-var-lib-kubelet\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518965    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-cni-bin\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.518989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4jh\" (UniqueName: \"kubernetes.io/projected/e4892ce1-b232-4ced-b142-289d7804ecff-kube-api-access-tf4jh\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-cni-binary-copy\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519052    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-cni-bin\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519082    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-cni-bin\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519110    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-kubelet\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519184    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519214    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-run-netns\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-slash\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519259    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovnkube-config\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519266    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-etc-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cnibin\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-run-netns\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519319    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-daemon-config\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-host-slash\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519337    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4cb01e07-87f3-42a4-a130-4110a6245b5e-etc-openvswitch\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519345    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-modprobe-d\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.520732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.519382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-host\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.521211 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.521189    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd8e5c13-07a9-49bd-bf9f-08de8be07870-agent-certs\") pod \"konnectivity-agent-r9ckt\" (UID: \"dd8e5c13-07a9-49bd-bf9f-08de8be07870\") " pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.521701 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.521685    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4cb01e07-87f3-42a4-a130-4110a6245b5e-ovn-node-metrics-cert\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.533355 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.533334    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxfjd\" (UniqueName: \"kubernetes.io/projected/ea544a8b-c838-491a-86aa-bc0c301fa723-kube-api-access-vxfjd\") pod \"iptables-alerter-rpn7m\" (UID: \"ea544a8b-c838-491a-86aa-bc0c301fa723\") " pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.533487 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.533469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwb2\" (UniqueName: \"kubernetes.io/projected/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-kube-api-access-kdwb2\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:02.533662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.533623    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54h9x\" (UniqueName: \"kubernetes.io/projected/4cb01e07-87f3-42a4-a130-4110a6245b5e-kube-api-access-54h9x\") pod \"ovnkube-node-qczpg\" (UID: \"4cb01e07-87f3-42a4-a130-4110a6245b5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.619715 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619677    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-device-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-system-cni-dir\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619749    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-os-release\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619766    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-run\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619791    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-os-release\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619803    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-device-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619808    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-system-cni-dir\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619811    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kxbb\" (UniqueName: \"kubernetes.io/projected/3887f0b3-3a95-42c6-943b-d945d53e8ac9-kube-api-access-4kxbb\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619846    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-os-release\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619838    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-run\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.619881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619876    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psbnx\" (UniqueName: \"kubernetes.io/projected/c65d0345-d0cf-40c2-9bd7-0086726a8f31-kube-api-access-psbnx\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-os-release\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.619945    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ed095f6-a14d-4e47-8778-d650a46ade19-tmp\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-systemd\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620162    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620178    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysctl-d\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620197    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c24b7c8-f986-4410-bf06-5d31d80f43b6-kubelet-config\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620201    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-systemd\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620212    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c24b7c8-f986-4410-bf06-5d31d80f43b6-dbus\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620249    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620253    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-multus-certs\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620268    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c24b7c8-f986-4410-bf06-5d31d80f43b6-kubelet-config\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620290    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-sys\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620322    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c24b7c8-f986-4410-bf06-5d31d80f43b6-dbus\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-multus-certs\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620330    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysctl-d\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.620357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-cni-multus\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-sys\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620388    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c65d0345-d0cf-40c2-9bd7-0086726a8f31-host\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620385    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-cni-multus\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysctl-conf\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620446    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c65d0345-d0cf-40c2-9bd7-0086726a8f31-host\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-hostroot\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620479    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-hostroot\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-system-cni-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620514    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-cni-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620530    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-var-lib-kubelet\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620548    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4jh\" (UniqueName: \"kubernetes.io/projected/e4892ce1-b232-4ced-b142-289d7804ecff-kube-api-access-tf4jh\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.620557    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620584    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysctl-conf\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-cni-binary-copy\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620618    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-cni-bin\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620620    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-cni-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621092 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.620630    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:03.120611926 +0000 UTC m=+3.085511512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620669    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-cni-bin\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-kubelet\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620706    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-var-lib-kubelet\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620710    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620748    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cnibin\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-daemon-config\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620789    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cnibin\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620801    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-modprobe-d\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620825    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-host\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620855    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620880    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-cnibin\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620902    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-host\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620907    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-kubernetes\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620938    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-lib-modules\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620938    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-modprobe-d\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620951    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-kubernetes\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-cnibin\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.621996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-etc-kubernetes\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621046    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621049    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-lib-modules\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621088    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-registration-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621115    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-etc-kubernetes\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-registration-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621159    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-sys-fs\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621199    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpb6x\" (UniqueName: \"kubernetes.io/projected/6407cbcb-cfef-4674-8527-25513af84e1e-kube-api-access-jpb6x\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621210    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-sys-fs\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621216    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-var-lib-kubelet\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621227    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-netns\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621239    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-cni-binary-copy\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621256    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysconfig\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-netns\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621286    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e4892ce1-b232-4ced-b142-289d7804ecff-hosts-file\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.620662    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-system-cni-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-sysconfig\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621334    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-daemon-config\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.622819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-k8s-cni-cncf-io\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621340    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e4892ce1-b232-4ced-b142-289d7804ecff-hosts-file\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-host-run-k8s-cni-cncf-io\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621378    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-conf-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621434    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cni-binary-copy\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621437    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-conf-dir\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621500    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-tuned\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621575    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgkhr\" (UniqueName: \"kubernetes.io/projected/8ed095f6-a14d-4e47-8778-d650a46ade19-kube-api-access-pgkhr\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621662    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-socket-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621694    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621737    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621799    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c65d0345-d0cf-40c2-9bd7-0086726a8f31-serviceca\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621833    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6407cbcb-cfef-4674-8527-25513af84e1e-socket-dir\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621849    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4892ce1-b232-4ced-b142-289d7804ecff-tmp-dir\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.621972    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3887f0b3-3a95-42c6-943b-d945d53e8ac9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622001    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-socket-dir-parent\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.623705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622030    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cni-binary-copy\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.624483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622071    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fmhr\" (UniqueName: \"kubernetes.io/projected/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-kube-api-access-9fmhr\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.624483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622109    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-multus-socket-dir-parent\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.624483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622307    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.624483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622669    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c65d0345-d0cf-40c2-9bd7-0086726a8f31-serviceca\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.624483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622748    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4892ce1-b232-4ced-b142-289d7804ecff-tmp-dir\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.624483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.622758    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3887f0b3-3a95-42c6-943b-d945d53e8ac9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.625475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.624983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ed095f6-a14d-4e47-8778-d650a46ade19-tmp\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.625475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.625036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ed095f6-a14d-4e47-8778-d650a46ade19-etc-tuned\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.651993 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.651968    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kxbb\" (UniqueName: \"kubernetes.io/projected/3887f0b3-3a95-42c6-943b-d945d53e8ac9-kube-api-access-4kxbb\") pod \"multus-additional-cni-plugins-m6n8j\" (UID: \"3887f0b3-3a95-42c6-943b-d945d53e8ac9\") " pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.653754 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.653736    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgkhr\" (UniqueName: \"kubernetes.io/projected/8ed095f6-a14d-4e47-8778-d650a46ade19-kube-api-access-pgkhr\") pod \"tuned-wnlzz\" (UID: \"8ed095f6-a14d-4e47-8778-d650a46ade19\") " pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.658397 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.658374    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:02.658475 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.658402    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:02.658475 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.658417    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:02.658595 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:02.658496    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:03.158476173 +0000 UTC m=+3.123375778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:02.659441 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.659417    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psbnx\" (UniqueName: \"kubernetes.io/projected/c65d0345-d0cf-40c2-9bd7-0086726a8f31-kube-api-access-psbnx\") pod \"node-ca-7k5q8\" (UID: \"c65d0345-d0cf-40c2-9bd7-0086726a8f31\") " pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.660339 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.660322    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fmhr\" (UniqueName: \"kubernetes.io/projected/e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3-kube-api-access-9fmhr\") pod \"multus-dp8bc\" (UID: \"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3\") " pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.660747 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.660724    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpb6x\" (UniqueName: \"kubernetes.io/projected/6407cbcb-cfef-4674-8527-25513af84e1e-kube-api-access-jpb6x\") pod \"aws-ebs-csi-driver-node-8xml6\" (UID: \"6407cbcb-cfef-4674-8527-25513af84e1e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.662906 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.662889    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4jh\" (UniqueName: \"kubernetes.io/projected/e4892ce1-b232-4ced-b142-289d7804ecff-kube-api-access-tf4jh\") pod \"node-resolver-kns2f\" (UID: \"e4892ce1-b232-4ced-b142-289d7804ecff\") " pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.707310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.707289    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:02.713897 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.713875    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rpn7m"
Jul 21 17:02:02.718392 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.718367    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8e5c13_07a9_49bd_bf9f_08de8be07870.slice/crio-95fa3cce8364dd6afaf3cf63abb134fb3c8fd091d6bcd329d5c8204a8a32b04a WatchSource:0}: Error finding container 95fa3cce8364dd6afaf3cf63abb134fb3c8fd091d6bcd329d5c8204a8a32b04a: Status 404 returned error can't find the container with id 95fa3cce8364dd6afaf3cf63abb134fb3c8fd091d6bcd329d5c8204a8a32b04a
Jul 21 17:02:02.723249 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.723226    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea544a8b_c838_491a_86aa_bc0c301fa723.slice/crio-e5f0e6a063ba0f8f1272638f753ae60578410dab822b92fdda64cd194e5ccbe4 WatchSource:0}: Error finding container e5f0e6a063ba0f8f1272638f753ae60578410dab822b92fdda64cd194e5ccbe4: Status 404 returned error can't find the container with id e5f0e6a063ba0f8f1272638f753ae60578410dab822b92fdda64cd194e5ccbe4
Jul 21 17:02:02.726467 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.726450    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:02.729800 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.729782    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6"
Jul 21 17:02:02.736305 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.736242    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kns2f"
Jul 21 17:02:02.739386 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.739255    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb01e07_87f3_42a4_a130_4110a6245b5e.slice/crio-ced45341a6481f00ad171b35edc55470eac2316096d284b58775cb7fbd160abc WatchSource:0}: Error finding container ced45341a6481f00ad171b35edc55470eac2316096d284b58775cb7fbd160abc: Status 404 returned error can't find the container with id ced45341a6481f00ad171b35edc55470eac2316096d284b58775cb7fbd160abc
Jul 21 17:02:02.740929 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.740905    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dp8bc"
Jul 21 17:02:02.743138 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.743117    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6407cbcb_cfef_4674_8527_25513af84e1e.slice/crio-0ecf29b35aa23c6756a76c11dda0ea46b7e5430e659b0791544da177bfbc9edb WatchSource:0}: Error finding container 0ecf29b35aa23c6756a76c11dda0ea46b7e5430e659b0791544da177bfbc9edb: Status 404 returned error can't find the container with id 0ecf29b35aa23c6756a76c11dda0ea46b7e5430e659b0791544da177bfbc9edb
Jul 21 17:02:02.746434 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.746414    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4892ce1_b232_4ced_b142_289d7804ecff.slice/crio-d52ce688b0bb9e54d772c857c3b9ac0c46707f9e153c985831d1c9510ebdacf7 WatchSource:0}: Error finding container d52ce688b0bb9e54d772c857c3b9ac0c46707f9e153c985831d1c9510ebdacf7: Status 404 returned error can't find the container with id d52ce688b0bb9e54d772c857c3b9ac0c46707f9e153c985831d1c9510ebdacf7
Jul 21 17:02:02.746917 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.746869    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m6n8j"
Jul 21 17:02:02.752048 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.752025    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c7149a_ab7e_4b82_aeb3_04d6be2bcab3.slice/crio-e1b3750f3b1092874cdb8ac624cbf738392f2e6ab830b88bfc98b3d7717d596b WatchSource:0}: Error finding container e1b3750f3b1092874cdb8ac624cbf738392f2e6ab830b88bfc98b3d7717d596b: Status 404 returned error can't find the container with id e1b3750f3b1092874cdb8ac624cbf738392f2e6ab830b88bfc98b3d7717d596b
Jul 21 17:02:02.752698 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.752683    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7k5q8"
Jul 21 17:02:02.756837 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.756815    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3887f0b3_3a95_42c6_943b_d945d53e8ac9.slice/crio-a386e67582b02c96ca5ca9b5c9649778ec04df0c9d198961971fc94551350d3d WatchSource:0}: Error finding container a386e67582b02c96ca5ca9b5c9649778ec04df0c9d198961971fc94551350d3d: Status 404 returned error can't find the container with id a386e67582b02c96ca5ca9b5c9649778ec04df0c9d198961971fc94551350d3d
Jul 21 17:02:02.757470 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:02.757445    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wnlzz"
Jul 21 17:02:02.762479 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.762455    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65d0345_d0cf_40c2_9bd7_0086726a8f31.slice/crio-79d6bd3a0e3dcb8a4a6cf922c828743bb62a9af7a755d901923b652403905c1d WatchSource:0}: Error finding container 79d6bd3a0e3dcb8a4a6cf922c828743bb62a9af7a755d901923b652403905c1d: Status 404 returned error can't find the container with id 79d6bd3a0e3dcb8a4a6cf922c828743bb62a9af7a755d901923b652403905c1d
Jul 21 17:02:02.767167 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:02.767146    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed095f6_a14d_4e47_8778_d650a46ade19.slice/crio-eef1faf004fe723dac21a4aee6d0002d8ca5832f130c710a8d6d35ffcca0158d WatchSource:0}: Error finding container eef1faf004fe723dac21a4aee6d0002d8ca5832f130c710a8d6d35ffcca0158d: Status 404 returned error can't find the container with id eef1faf004fe723dac21a4aee6d0002d8ca5832f130c710a8d6d35ffcca0158d
Jul 21 17:02:03.025135 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.025050    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:03.025376 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.025224    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:03.025376 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.025293    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:02:04.025272398 +0000 UTC m=+3.990171988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:03.125838 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.125803    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:03.126009 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.125964    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:03.126083 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.126032    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:04.126012592 +0000 UTC m=+4.090912201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:03.227225 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.227189    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:03.227463 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.227442    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:03.227520 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.227475    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:03.227520 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.227489    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:03.227625 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:03.227552    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:04.227537158 +0000 UTC m=+4.192436741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:03.512885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.512833    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 16:57:02 +0000 UTC" deadline="2028-04-16 17:12:17.165167592 +0000 UTC"
Jul 21 17:02:03.512885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.512881    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15240h10m13.652291445s"
Jul 21 17:02:03.521016 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.520976    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7k5q8" event={"ID":"c65d0345-d0cf-40c2-9bd7-0086726a8f31","Type":"ContainerStarted","Data":"79d6bd3a0e3dcb8a4a6cf922c828743bb62a9af7a755d901923b652403905c1d"}
Jul 21 17:02:03.536153 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.536123    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dp8bc" event={"ID":"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3","Type":"ContainerStarted","Data":"e1b3750f3b1092874cdb8ac624cbf738392f2e6ab830b88bfc98b3d7717d596b"}
Jul 21 17:02:03.541996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.541963    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"ced45341a6481f00ad171b35edc55470eac2316096d284b58775cb7fbd160abc"}
Jul 21 17:02:03.550782 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.550678    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rpn7m" event={"ID":"ea544a8b-c838-491a-86aa-bc0c301fa723","Type":"ContainerStarted","Data":"e5f0e6a063ba0f8f1272638f753ae60578410dab822b92fdda64cd194e5ccbe4"}
Jul 21 17:02:03.556872 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.556841    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerStarted","Data":"a386e67582b02c96ca5ca9b5c9649778ec04df0c9d198961971fc94551350d3d"}
Jul 21 17:02:03.567378 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.567347    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kns2f" event={"ID":"e4892ce1-b232-4ced-b142-289d7804ecff","Type":"ContainerStarted","Data":"d52ce688b0bb9e54d772c857c3b9ac0c46707f9e153c985831d1c9510ebdacf7"}
Jul 21 17:02:03.576752 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.576720    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6" event={"ID":"6407cbcb-cfef-4674-8527-25513af84e1e","Type":"ContainerStarted","Data":"0ecf29b35aa23c6756a76c11dda0ea46b7e5430e659b0791544da177bfbc9edb"}
Jul 21 17:02:03.584932 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.584898    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r9ckt" event={"ID":"dd8e5c13-07a9-49bd-bf9f-08de8be07870","Type":"ContainerStarted","Data":"95fa3cce8364dd6afaf3cf63abb134fb3c8fd091d6bcd329d5c8204a8a32b04a"}
Jul 21 17:02:03.589417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:03.589387    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wnlzz" event={"ID":"8ed095f6-a14d-4e47-8778-d650a46ade19","Type":"ContainerStarted","Data":"eef1faf004fe723dac21a4aee6d0002d8ca5832f130c710a8d6d35ffcca0158d"}
Jul 21 17:02:04.037952 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.037831    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:04.038129 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.038002    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:04.038129 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.038080    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:02:06.038056599 +0000 UTC m=+6.002956204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:04.138834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.138797    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:04.139007 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.138953    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:04.139084 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.139015    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:06.138995478 +0000 UTC m=+6.103895062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:04.240086 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.240048    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:04.240279 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.240218    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:04.240279 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.240237    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:04.240279 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.240251    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:04.240459 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.240311    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:06.240291384 +0000 UTC m=+6.205190973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:04.510265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.509353    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:04.510265 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.509493    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:04.510265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.509817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:04.510265 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.509924    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:04.510265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.509937    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:04.510265 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:04.510027    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:04.513597 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.513204    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 16:57:02 +0000 UTC" deadline="2028-03-14 10:25:54.72018154 +0000 UTC"
Jul 21 17:02:04.513597 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:04.513268    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14441h23m50.206917932s"
Jul 21 17:02:06.056372 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:06.055739    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:06.056372 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.055949    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:06.056372 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.056013    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:02:10.055993674 +0000 UTC m=+10.020893263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:06.156401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:06.156362    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:06.156629 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.156573    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:06.156715 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.156655    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:10.15663576 +0000 UTC m=+10.121535345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:06.257295 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:06.257257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:06.257499 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.257483    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:06.257568 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.257508    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:06.257568 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.257526    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:06.257568 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.257584    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:10.257564954 +0000 UTC m=+10.222464543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:06.510383 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:06.510300    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:06.510542 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.510436    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:06.510884 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:06.510862    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:06.511014 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.510991    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:06.511103 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:06.511059    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:06.511160 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:06.511130    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:08.509592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:08.508906    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:08.509592 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:08.509041    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:08.509592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:08.509236    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:08.509592 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:08.509324    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:08.509592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:08.509369    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:08.509592 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:08.509438    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:10.093895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:10.093857    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:10.094321 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.094043    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:10.094321 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.094133    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:02:18.094104905 +0000 UTC m=+18.059004501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:10.194599 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:10.194495    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:10.194761 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.194653    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:10.194761 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.194724    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:18.194703195 +0000 UTC m=+18.159602791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:10.295865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:10.295249    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:10.295865 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.295452    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:10.295865 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.295468    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:10.295865 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.295479    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:10.295865 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.295531    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:18.295514507 +0000 UTC m=+18.260414092 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:10.508111 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:10.508078    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:10.508292 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.508209    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:10.508441 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:10.508409    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:10.508533 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.508509    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:10.508533 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:10.508081    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:10.508661 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:10.508646    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:12.507187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:12.507154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:12.507612 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:12.507203    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:12.507612 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:12.507286    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:12.507612 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:12.507321    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:12.507612 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:12.507449    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:12.507612 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:12.507537    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:14.507743 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:14.507254    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:14.507743 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:14.507264    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:14.507743 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:14.507370    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:14.507743 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:14.507393    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:14.507743 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:14.507527    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:14.507743 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:14.507664    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:16.507966 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:16.507929    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:16.508412 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:16.507929    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:16.508412 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:16.508061    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:16.508412 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:16.508164    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:16.508412 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:16.507939    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:16.508412 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:16.508256    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:18.155205 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:18.155155    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:18.155627 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.155316    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:18.155627 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.155371    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:02:34.155356509 +0000 UTC m=+34.120256091 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:18.255967 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:18.255929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:18.256171 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.256132    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:18.256237 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.256213    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:34.256190284 +0000 UTC m=+34.221089874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:18.356638 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:18.356606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:18.356820 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.356767    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:18.356820 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.356790    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:18.356820 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.356802    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:18.356980 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.356867    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:02:34.356846253 +0000 UTC m=+34.321745837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:18.507442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:18.507366    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:18.507442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:18.507393    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:18.507442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:18.507366    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:18.507756 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.507519    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:18.507756 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.507665    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:18.508545 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:18.508095    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:20.509985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:20.509786    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:20.510422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:20.509851    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:20.510422 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:20.510082    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:20.510422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:20.509851    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:20.510422 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:20.510167    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:20.510422 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:20.510240    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:22.510611 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:22.510579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:22.511136 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:22.510579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:22.511136 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:22.510699    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:22.511136 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:22.510794    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:22.511136 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:22.510585    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:22.511136 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:22.510911    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:23.638855 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.638822    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal" event={"ID":"de9305d86924dd928dcf28542838bdc5","Type":"ContainerStarted","Data":"c5952cc4bb2eb98d3426ca1af7da8ef40c4ff7552405c135deb22e4682fbdd85"}
Jul 21 17:02:23.647865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.647838    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wnlzz" event={"ID":"8ed095f6-a14d-4e47-8778-d650a46ade19","Type":"ContainerStarted","Data":"f90f1c7fc33868df8878ad981da2bb563d49bb0fe0fde6bf2ff79579c515e72a"}
Jul 21 17:02:23.649428 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.649403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dp8bc" event={"ID":"e9c7149a-ab7e-4b82-aeb3-04d6be2bcab3","Type":"ContainerStarted","Data":"3c6d4ce952d3ef71cf1be5a5572f8f285c88350dfe070a54277f8b266e4cfd05"}
Jul 21 17:02:23.651633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.651492    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"09d9567feaf807812408c1b70607fe02fe2116aee4670d26f8ed88b6bdc920c8"}
Jul 21 17:02:23.651730 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.651673    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"6a759d1b1139798cdebe8e0b27060a478779e67c7429358a566ac859fa264630"}
Jul 21 17:02:23.651784 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.651728    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"99579e23d3d903989d32611b5105ed1d9d5285c3761178bc2bb3e61c4822feaa"}
Jul 21 17:02:23.683987 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.683948    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-24.ec2.internal" podStartSLOduration=21.683930307 podStartE2EDuration="21.683930307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:23.660257155 +0000 UTC m=+23.625156762" watchObservedRunningTime="2026-07-21 17:02:23.683930307 +0000 UTC m=+23.648829910"
Jul 21 17:02:23.705585 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.705527    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dp8bc" podStartSLOduration=2.227340763 podStartE2EDuration="22.705512915s" totalImagesPullingTime="20.478172152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:23.705177487 +0000 UTC m=+23.670077091" watchObservedRunningTime="2026-07-21 17:02:23.705512915 +0000 UTC m=+23.670412563"
Jul 21 17:02:23.705858 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:23.705829    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wnlzz" podStartSLOduration=2.278900116 podStartE2EDuration="22.705821427s" totalImagesPullingTime="20.426921311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:23.684067338 +0000 UTC m=+23.648966933" watchObservedRunningTime="2026-07-21 17:02:23.705821427 +0000 UTC m=+23.670721031"
Jul 21 17:02:24.508212 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.507962    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:24.508343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.507977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:24.508343 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:24.508291    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:24.508343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.508023    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:24.508498 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:24.508353    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:24.508498 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:24.508446    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:24.655347 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.655317    2561 generic.go:356] "Generic (PLEG): container finished" podID="3e7b7687333f4ca94a71843379ec67a5" containerID="d94dd4a177417cfa01dc2e7cbaccfbdcfc2749afc4b007eef3f93300ee728914" exitCode=0
Jul 21 17:02:24.656121 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.655423    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal" event={"ID":"3e7b7687333f4ca94a71843379ec67a5","Type":"ContainerDied","Data":"d94dd4a177417cfa01dc2e7cbaccfbdcfc2749afc4b007eef3f93300ee728914"}
Jul 21 17:02:24.657333 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.657123    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7k5q8" event={"ID":"c65d0345-d0cf-40c2-9bd7-0086726a8f31","Type":"ContainerStarted","Data":"b64bbbaf348c39fa7e7b6456287208f0f7df19970010a703c37b087ef7bb10f1"}
Jul 21 17:02:24.660326 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.660306    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:02:24.660770 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.660746    2561 generic.go:356] "Generic (PLEG): container finished" podID="4cb01e07-87f3-42a4-a130-4110a6245b5e" containerID="6a759d1b1139798cdebe8e0b27060a478779e67c7429358a566ac859fa264630" exitCode=1
Jul 21 17:02:24.660892 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.660830    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerDied","Data":"6a759d1b1139798cdebe8e0b27060a478779e67c7429358a566ac859fa264630"}
Jul 21 17:02:24.660948 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.660900    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"ed2f94c9107b213afcabc5d0d002d1e7dcf0875a3b64b68930b6f4f7c03c2906"}
Jul 21 17:02:24.661000 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.660956    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"814ff76abd4744bf9d54bf28fadd5e2c9aa91712d42e88d3d0a207f9409cc5dd"}
Jul 21 17:02:24.661049 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.661005    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"719182514b3bcb06cb0433c2de07474ea49e38ab808d38ae53a421c06d9c310a"}
Jul 21 17:02:24.662615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.662591    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rpn7m" event={"ID":"ea544a8b-c838-491a-86aa-bc0c301fa723","Type":"ContainerStarted","Data":"904c3d925a40bd068f6dac288300164e30b444183494a8d7ede9ee14af6f88d7"}
Jul 21 17:02:24.664187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.664165    2561 generic.go:356] "Generic (PLEG): container finished" podID="3887f0b3-3a95-42c6-943b-d945d53e8ac9" containerID="d28f2124e28a8a6ba2e1ed7f810cb8a2f48289fa5f794cd4450cf053f0688e57" exitCode=0
Jul 21 17:02:24.664287 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.664239    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerDied","Data":"d28f2124e28a8a6ba2e1ed7f810cb8a2f48289fa5f794cd4450cf053f0688e57"}
Jul 21 17:02:24.666242 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.666212    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kns2f" event={"ID":"e4892ce1-b232-4ced-b142-289d7804ecff","Type":"ContainerStarted","Data":"9dddcc3904682f11f81ffc563463085d073cb7a103d1e6a49b81aa9d629222d2"}
Jul 21 17:02:24.667860 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.667838    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6" event={"ID":"6407cbcb-cfef-4674-8527-25513af84e1e","Type":"ContainerStarted","Data":"d6738687c5f0b2a404557b66484448a498aefe781da19652bdbb12e665be8fb0"}
Jul 21 17:02:24.671163 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.671138    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r9ckt" event={"ID":"dd8e5c13-07a9-49bd-bf9f-08de8be07870","Type":"ContainerStarted","Data":"0f56ae53865f7e276787bdc94ebd3bd8f91aff2c0f74463df702df571b5087bc"}
Jul 21 17:02:24.724399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.724352    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rpn7m" podStartSLOduration=3.588682728 podStartE2EDuration="23.724338099s" totalImagesPullingTime="20.135655371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:24.692948541 +0000 UTC m=+24.657848144" watchObservedRunningTime="2026-07-21 17:02:24.724338099 +0000 UTC m=+24.689237703"
Jul 21 17:02:24.765216 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.765115    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7k5q8" podStartSLOduration=3.360042823 podStartE2EDuration="23.765100414s" totalImagesPullingTime="20.405057591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:24.742148982 +0000 UTC m=+24.707048586" watchObservedRunningTime="2026-07-21 17:02:24.765100414 +0000 UTC m=+24.730000014"
Jul 21 17:02:24.789417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.789380    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kns2f" podStartSLOduration=3.366883507 podStartE2EDuration="23.789366351s" totalImagesPullingTime="20.422482844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:24.764775823 +0000 UTC m=+24.729675428" watchObservedRunningTime="2026-07-21 17:02:24.789366351 +0000 UTC m=+24.754265955"
Jul 21 17:02:24.902780 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.902742    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:24.974930 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:24.974894    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:25.310019 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.309998    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 17:02:25.490412 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.490293    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T17:02:25.310014299Z","UUID":"15b8f2d5-e35c-4829-8acb-3a0318423e8d","Handler":null,"Name":"","Endpoint":""}
Jul 21 17:02:25.491618 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.491599    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 17:02:25.491740 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.491637    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 17:02:25.675222 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.675189    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6" event={"ID":"6407cbcb-cfef-4674-8527-25513af84e1e","Type":"ContainerStarted","Data":"dd9d92dd8b2400b8eccffbb4d9bb585d05558dcc30c4dfb854b784f5b95bce17"}
Jul 21 17:02:25.677083 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.677049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal" event={"ID":"3e7b7687333f4ca94a71843379ec67a5","Type":"ContainerStarted","Data":"d167d34d60b7f3d4fcdb2fd2e9fdd1292a06b2e835680b9000a25bd31ed9b321"}
Jul 21 17:02:25.718131 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.718081    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-24.ec2.internal" podStartSLOduration=23.718067381 podStartE2EDuration="23.718067381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:25.718011933 +0000 UTC m=+25.682911536" watchObservedRunningTime="2026-07-21 17:02:25.718067381 +0000 UTC m=+25.682966984"
Jul 21 17:02:25.718372 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:25.718352    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r9ckt" podStartSLOduration=8.431307602 podStartE2EDuration="24.718344415s" totalImagesPullingTime="16.287036813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:24.789349689 +0000 UTC m=+24.754249292" watchObservedRunningTime="2026-07-21 17:02:25.718344415 +0000 UTC m=+25.683244019"
Jul 21 17:02:26.507593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:26.507548    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:26.507791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:26.507595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:26.507791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:26.507676    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:26.507791 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:26.507692    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:26.507952 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:26.507802    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:26.507952 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:26.507924    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:26.684865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:26.684835    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:02:26.685402 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:26.685323    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"f114de4f0ecdee876b720b40d3cfdca8e3678c9d838cb2c444bff524609cf8e4"}
Jul 21 17:02:27.689911 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:27.689878    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6" event={"ID":"6407cbcb-cfef-4674-8527-25513af84e1e","Type":"ContainerStarted","Data":"f3ef722afd90a992839a4d86c3ba90e6b7525db9011a93e1481d35d867d058af"}
Jul 21 17:02:27.711806 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:27.711755    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xml6" podStartSLOduration=3.251312866 podStartE2EDuration="26.711737256s" totalImagesPullingTime="23.46042439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:27.711509349 +0000 UTC m=+27.676408966" watchObservedRunningTime="2026-07-21 17:02:27.711737256 +0000 UTC m=+27.676636861"
Jul 21 17:02:28.508017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:28.507933    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:28.508017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:28.507969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:28.508017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:28.507934    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:28.508284 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:28.508072    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:28.508284 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:28.508160    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:28.508284 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:28.508265    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:28.698387 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:28.698349    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:28.699133 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:28.699114    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:29.244167 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.244131    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:29.324989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.324958    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:29.486685 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.486649    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:29.560440 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.560337    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:29.638683 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.638631    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:29.694607 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.694526    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:29.695113 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.695089    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r9ckt"
Jul 21 17:02:29.773962 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.773932    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:29.874288 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:29.874185    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:30.032941 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.032902    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:30.136465 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.136422    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:30.507970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.507941    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:30.508087 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.508067    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:30.508087 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:30.508069    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:30.508232 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:30.508195    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:30.508291 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.508275    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:30.508368 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:30.508351    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:30.604896 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.604874    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kns2f_e4892ce1-b232-4ced-b142-289d7804ecff/dns-node-resolver/0.log"
Jul 21 17:02:30.699839 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.699817    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:02:30.700205 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.700184    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"4dbceb2755041341cea446fcba7ccda3a6ea234f6dc58546670b2c6f77349b8f"}
Jul 21 17:02:30.700508 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.700476    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:30.700646 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.700623    2561 scope.go:117] "RemoveContainer" containerID="6a759d1b1139798cdebe8e0b27060a478779e67c7429358a566ac859fa264630"
Jul 21 17:02:30.702162 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.702141    2561 generic.go:356] "Generic (PLEG): container finished" podID="3887f0b3-3a95-42c6-943b-d945d53e8ac9" containerID="3e3322d527bd3bdf902b5eb11dabad5fcaf3533894200248726ff4d487fc09e7" exitCode=0
Jul 21 17:02:30.702264 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.702237    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerDied","Data":"3e3322d527bd3bdf902b5eb11dabad5fcaf3533894200248726ff4d487fc09e7"}
Jul 21 17:02:30.716673 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:30.716654    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:31.022989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.022764    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:31.122144 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.122099    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:31.404084 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.404061    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7k5q8_c65d0345-d0cf-40c2-9bd7-0086726a8f31/node-ca/0.log"
Jul 21 17:02:31.708724 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.708665    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:02:31.709080 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.709048    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" event={"ID":"4cb01e07-87f3-42a4-a130-4110a6245b5e","Type":"ContainerStarted","Data":"3f3f7cd910d86ac294cd20077b35d52f0730c853018c2c7327077731f4656ca1"}
Jul 21 17:02:31.709287 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.709269    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:31.709409 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.709390    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:31.725350 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.725326    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:02:31.755945 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:31.755900    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg" podStartSLOduration=10.263906168 podStartE2EDuration="30.755885818s" totalImagesPullingTime="20.49197965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:31.754404185 +0000 UTC m=+31.719303790" watchObservedRunningTime="2026-07-21 17:02:31.755885818 +0000 UTC m=+31.720785493"
Jul 21 17:02:32.077667 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.077156    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d6h64"]
Jul 21 17:02:32.077667 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.077279    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:32.077667 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:32.077357    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:32.087675 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.087645    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wpvhl"]
Jul 21 17:02:32.087859 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.087775    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:32.087923 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:32.087884    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:32.121682 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.121653    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jjmrd"]
Jul 21 17:02:32.121837 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.121779    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:32.121940 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:32.121919    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:32.244076 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.244040    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:32.508395 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.508365    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:32.658926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.658894    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:32.714345 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.714317    2561 generic.go:356] "Generic (PLEG): container finished" podID="3887f0b3-3a95-42c6-943b-d945d53e8ac9" containerID="240d5b18c09ca5160fcad4753c2efd016ee6dc4bf7f138979f7b144c3720df80" exitCode=0
Jul 21 17:02:32.714510 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:32.714413    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerDied","Data":"240d5b18c09ca5160fcad4753c2efd016ee6dc4bf7f138979f7b144c3720df80"}
Jul 21 17:02:33.325404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:33.325373    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:33.507471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:33.507439    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:33.507622 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:33.507439    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:33.507622 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:33.507543    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:33.507695 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:33.507628    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:34.118393 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.118320    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:34.175803 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.175774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:34.175943 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.175914    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:34.175992 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.175984    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs podName:7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c nodeName:}" failed. No retries permitted until 2026-07-21 17:03:06.175968254 +0000 UTC m=+66.140867839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs") pod "network-metrics-daemon-wpvhl" (UID: "7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 17:02:34.216297 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.216269    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:34.276694 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.276664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:34.276832 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.276791    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:34.276890 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.276865    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret podName:0c24b7c8-f986-4410-bf06-5d31d80f43b6 nodeName:}" failed. No retries permitted until 2026-07-21 17:03:06.27684284 +0000 UTC m=+66.241742422 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret") pod "global-pull-secret-syncer-jjmrd" (UID: "0c24b7c8-f986-4410-bf06-5d31d80f43b6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 17:02:34.363680 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.363654    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:34.377451 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.377400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:34.377526 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.377519    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 17:02:34.377585 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.377531    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 17:02:34.377585 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.377539    2561 projected.go:196] Error preparing data for projected volume kube-api-access-b9gvd for pod openshift-network-diagnostics/network-check-target-d6h64: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:34.377648 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.377595    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd podName:849ea74a-b773-4c55-85b1-a891929c6e25 nodeName:}" failed. No retries permitted until 2026-07-21 17:03:06.377583852 +0000 UTC m=+66.342483434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b9gvd" (UniqueName: "kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd") pod "network-check-target-d6h64" (UID: "849ea74a-b773-4c55-85b1-a891929c6e25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 17:02:34.507550 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.507521    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:34.507708 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:34.507645    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:34.635631 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.635538    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log" limit=1024
Jul 21 17:02:34.721842 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.721814    2561 generic.go:356] "Generic (PLEG): container finished" podID="3887f0b3-3a95-42c6-943b-d945d53e8ac9" containerID="67bd77104e3d402c073f0a11b7d1d61b43382069cf7c40fba02036f2e725d47b" exitCode=0
Jul 21 17:02:34.721955 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:34.721865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerDied","Data":"67bd77104e3d402c073f0a11b7d1d61b43382069cf7c40fba02036f2e725d47b"}
Jul 21 17:02:35.507721 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:35.507692    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:35.508079 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:35.507799    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:35.508079 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:35.507854    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:35.508079 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:35.507938    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:36.507608 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:36.507574    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:36.507821 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:36.507703    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:37.507991 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:37.507959    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:37.508431 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:37.507966    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:37.508431 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:37.508083    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:37.508431 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:37.508133    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:38.507351 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:38.507313    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:38.507548 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:38.507461    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:39.507267 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:39.507225    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:39.507267 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:39.507270    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:39.507776 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:39.507399    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:39.507776 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:39.507550    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:40.508154 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:40.507997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:40.508489 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:40.508208    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:40.738595 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:40.738543    2561 generic.go:356] "Generic (PLEG): container finished" podID="3887f0b3-3a95-42c6-943b-d945d53e8ac9" containerID="97830899a35937f7f7c6e1cdfe78c6197e7fe6495e026590317080c60db0e745" exitCode=0
Jul 21 17:02:40.738764 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:40.738596    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerDied","Data":"97830899a35937f7f7c6e1cdfe78c6197e7fe6495e026590317080c60db0e745"}
Jul 21 17:02:41.507362 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:41.507327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:41.507586 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:41.507330    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:41.507586 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:41.507457    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:41.507586 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:41.507511    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:41.743607 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:41.743570    2561 generic.go:356] "Generic (PLEG): container finished" podID="3887f0b3-3a95-42c6-943b-d945d53e8ac9" containerID="199e80412b47d125ccb249957364b05b0be0a5bfff80855da5a519f4639422bf" exitCode=0
Jul 21 17:02:41.744052 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:41.743626    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerDied","Data":"199e80412b47d125ccb249957364b05b0be0a5bfff80855da5a519f4639422bf"}
Jul 21 17:02:42.507720 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:42.507685    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:42.507922 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:42.507827    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:42.748304 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:42.748263    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" event={"ID":"3887f0b3-3a95-42c6-943b-d945d53e8ac9","Type":"ContainerStarted","Data":"089008969b88a9f17231df06a7da7e81009c2f27cfa5e248b83a26b1d551dc63"}
Jul 21 17:02:42.777951 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:42.777849    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m6n8j" podStartSLOduration=7.923286941 podStartE2EDuration="41.777835511s" totalImagesPullingTime="33.85454857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:42.777655672 +0000 UTC m=+42.742555277" watchObservedRunningTime="2026-07-21 17:02:42.777835511 +0000 UTC m=+42.742735114"
Jul 21 17:02:43.507651 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:43.507620    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:43.507651 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:43.507658    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:43.507912 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:43.507742    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:43.507912 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:43.507876    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:44.507216 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:44.507178    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:44.507623 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:44.507326    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:45.507549 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:45.507521    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:45.507936 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:45.507649    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:45.507936 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:45.507708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:45.507936 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:45.507806    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:46.510017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:46.509987    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:46.510397 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:46.510091    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:47.507623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:47.507593    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:47.507937 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:47.507687    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:47.507937 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:47.507744    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:47.507937 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:47.507821    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:48.507547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:48.507517    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:48.507968 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:48.507645    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:49.507509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:49.507475    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:49.507694 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:49.507475    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:49.507694 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:49.507612    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:49.507694 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:49.507666    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:50.510359 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:50.510334    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:50.510756 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:50.510430    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:51.507296 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:51.507263    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:51.507483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:51.507263    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:51.507483 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:51.507364    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:51.507483 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:51.507452    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:52.510190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:52.510159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:52.510652 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:52.510258    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:53.507922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:53.507890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:53.508161 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:53.507988    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjmrd" podUID="0c24b7c8-f986-4410-bf06-5d31d80f43b6"
Jul 21 17:02:53.508161 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:53.508045    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:53.508161 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:53.508138    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6h64" podUID="849ea74a-b773-4c55-85b1-a891929c6e25"
Jul 21 17:02:54.510000 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:54.509975    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:54.510404 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:54.510075    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpvhl" podUID="7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c"
Jul 21 17:02:55.314504 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.314468    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-24.ec2.internal" event="NodeReady"
Jul 21 17:02:55.316861 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.315189    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 17:02:55.368439 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.368406    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b8bb8d44-wvlbf"]
Jul 21 17:02:55.384734 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.384708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.388085 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.388066    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 17:02:55.388802 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.388771    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 17:02:55.388909 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.388812    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8bb8d44-wvlbf"]
Jul 21 17:02:55.388909 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.388825    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fjqct\""
Jul 21 17:02:55.389208 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.389190    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 17:02:55.392779 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.392761    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2pcgv"]
Jul 21 17:02:55.404481 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.404455    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 17:02:55.414535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.414512    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gmllb"]
Jul 21 17:02:55.414661 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.414645    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.417339 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.417323    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 17:02:55.417540 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.417525    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bhhmq\""
Jul 21 17:02:55.417646 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.417620    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 17:02:55.430996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.430978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/100e03cb-0ee7-4983-810d-0906466049c2-image-registry-private-configuration\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-registry-tls\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/100e03cb-0ee7-4983-810d-0906466049c2-installation-pull-secrets\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431056    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-bound-sa-token\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431191 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/100e03cb-0ee7-4983-810d-0906466049c2-ca-trust-extracted\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431191 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/100e03cb-0ee7-4983-810d-0906466049c2-registry-certificates\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431191 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431167    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/100e03cb-0ee7-4983-810d-0906466049c2-trusted-ca\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.431191 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.431182    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szns8\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-kube-api-access-szns8\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.440844 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.440826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.440844 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.440837    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2pcgv"]
Jul 21 17:02:55.440960 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.440884    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gmllb"]
Jul 21 17:02:55.443072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.443052    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 17:02:55.443158 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.443096    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 17:02:55.443158 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.443063    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntxgg\""
Jul 21 17:02:55.443158 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.443152    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 17:02:55.443326 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.443288    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 17:02:55.484780 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.484757    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-grf9q"]
Jul 21 17:02:55.498686 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.498669    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.501217 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.501195    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-grf9q"]
Jul 21 17:02:55.501545 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.501528    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 17:02:55.501689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.501646    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4j6ls\""
Jul 21 17:02:55.501689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.501657    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 17:02:55.501689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.501683    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 17:02:55.506999 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.506978    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:02:55.507157 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.507141    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:02:55.510110 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.510083    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-59btl\""
Jul 21 17:02:55.510389 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.510329    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 17:02:55.510389 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.510384    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 17:02:55.510459 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.510405    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 17:02:55.531903 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.531878    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sbnc\" (UniqueName: \"kubernetes.io/projected/1868e074-fca4-47b5-a4e2-87e2eb73923d-kube-api-access-2sbnc\") pod \"ingress-canary-grf9q\" (UID: \"1868e074-fca4-47b5-a4e2-87e2eb73923d\") " pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.531997 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.531909    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/100e03cb-0ee7-4983-810d-0906466049c2-installation-pull-secrets\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.531997 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.531927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b75e216-b714-4036-8214-ca7adddb4839-config-volume\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.531997 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.531966    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-bound-sa-token\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532109 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.531996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2csxg\" (UniqueName: \"kubernetes.io/projected/24734282-d9c9-411e-b08b-748d83bc3fe4-kube-api-access-2csxg\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.532109 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532031    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbx8x\" (UniqueName: \"kubernetes.io/projected/9b75e216-b714-4036-8214-ca7adddb4839-kube-api-access-nbx8x\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.532109 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24734282-d9c9-411e-b08b-748d83bc3fe4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.532109 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532104    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/100e03cb-0ee7-4983-810d-0906466049c2-ca-trust-extracted\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532131    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/100e03cb-0ee7-4983-810d-0906466049c2-registry-certificates\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/100e03cb-0ee7-4983-810d-0906466049c2-trusted-ca\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532181    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szns8\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-kube-api-access-szns8\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532201    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24734282-d9c9-411e-b08b-748d83bc3fe4-crio-socket\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.532430 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532279    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/100e03cb-0ee7-4983-810d-0906466049c2-image-registry-private-configuration\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532648 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532627    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-registry-tls\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.532725 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532668    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1868e074-fca4-47b5-a4e2-87e2eb73923d-cert\") pod \"ingress-canary-grf9q\" (UID: \"1868e074-fca4-47b5-a4e2-87e2eb73923d\") " pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.532725 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532706    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24734282-d9c9-411e-b08b-748d83bc3fe4-data-volume\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.532826 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532737    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24734282-d9c9-411e-b08b-748d83bc3fe4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.532826 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532770    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b75e216-b714-4036-8214-ca7adddb4839-metrics-tls\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.532826 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.532793    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b75e216-b714-4036-8214-ca7adddb4839-tmp-dir\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.533192 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.533170    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/100e03cb-0ee7-4983-810d-0906466049c2-ca-trust-extracted\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.533464 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.533437    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/100e03cb-0ee7-4983-810d-0906466049c2-registry-certificates\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.533821 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.533796    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/100e03cb-0ee7-4983-810d-0906466049c2-trusted-ca\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.537980 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.537608    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/100e03cb-0ee7-4983-810d-0906466049c2-image-registry-private-configuration\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.537980 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.537974    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-registry-tls\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.538115 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.537612    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/100e03cb-0ee7-4983-810d-0906466049c2-installation-pull-secrets\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.541459 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.541442    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-bound-sa-token\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.546660 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.546639    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szns8\" (UniqueName: \"kubernetes.io/projected/100e03cb-0ee7-4983-810d-0906466049c2-kube-api-access-szns8\") pod \"image-registry-5b8bb8d44-wvlbf\" (UID: \"100e03cb-0ee7-4983-810d-0906466049c2\") " pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.633742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24734282-d9c9-411e-b08b-748d83bc3fe4-crio-socket\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.633742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633710    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1868e074-fca4-47b5-a4e2-87e2eb73923d-cert\") pod \"ingress-canary-grf9q\" (UID: \"1868e074-fca4-47b5-a4e2-87e2eb73923d\") " pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.633742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24734282-d9c9-411e-b08b-748d83bc3fe4-data-volume\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.633742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24734282-d9c9-411e-b08b-748d83bc3fe4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633764    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b75e216-b714-4036-8214-ca7adddb4839-metrics-tls\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633785    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b75e216-b714-4036-8214-ca7adddb4839-tmp-dir\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633813    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sbnc\" (UniqueName: \"kubernetes.io/projected/1868e074-fca4-47b5-a4e2-87e2eb73923d-kube-api-access-2sbnc\") pod \"ingress-canary-grf9q\" (UID: \"1868e074-fca4-47b5-a4e2-87e2eb73923d\") " pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633839    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b75e216-b714-4036-8214-ca7adddb4839-config-volume\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633878    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2csxg\" (UniqueName: \"kubernetes.io/projected/24734282-d9c9-411e-b08b-748d83bc3fe4-kube-api-access-2csxg\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633886    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24734282-d9c9-411e-b08b-748d83bc3fe4-crio-socket\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbx8x\" (UniqueName: \"kubernetes.io/projected/9b75e216-b714-4036-8214-ca7adddb4839-kube-api-access-nbx8x\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.634045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.633941    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24734282-d9c9-411e-b08b-748d83bc3fe4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.634422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.634131    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24734282-d9c9-411e-b08b-748d83bc3fe4-data-volume\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.634422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.634167    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b75e216-b714-4036-8214-ca7adddb4839-tmp-dir\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.634537 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.634431    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b75e216-b714-4036-8214-ca7adddb4839-config-volume\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.634537 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.634519    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24734282-d9c9-411e-b08b-748d83bc3fe4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.636421 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.636400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b75e216-b714-4036-8214-ca7adddb4839-metrics-tls\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.636837 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.636816    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24734282-d9c9-411e-b08b-748d83bc3fe4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.636922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.636908    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1868e074-fca4-47b5-a4e2-87e2eb73923d-cert\") pod \"ingress-canary-grf9q\" (UID: \"1868e074-fca4-47b5-a4e2-87e2eb73923d\") " pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.644325 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.644294    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbx8x\" (UniqueName: \"kubernetes.io/projected/9b75e216-b714-4036-8214-ca7adddb4839-kube-api-access-nbx8x\") pod \"dns-default-2pcgv\" (UID: \"9b75e216-b714-4036-8214-ca7adddb4839\") " pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.644435 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.644419    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2csxg\" (UniqueName: \"kubernetes.io/projected/24734282-d9c9-411e-b08b-748d83bc3fe4-kube-api-access-2csxg\") pod \"insights-runtime-extractor-gmllb\" (UID: \"24734282-d9c9-411e-b08b-748d83bc3fe4\") " pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.644586 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.644548    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sbnc\" (UniqueName: \"kubernetes.io/projected/1868e074-fca4-47b5-a4e2-87e2eb73923d-kube-api-access-2sbnc\") pod \"ingress-canary-grf9q\" (UID: \"1868e074-fca4-47b5-a4e2-87e2eb73923d\") " pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.695426 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.695396    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:55.722717 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.722689    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:02:55.749263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.749240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gmllb"
Jul 21 17:02:55.808288 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.808097    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-grf9q"
Jul 21 17:02:55.926548 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.926521    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2pcgv"]
Jul 21 17:02:55.940000 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.939977    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8bb8d44-wvlbf"]
Jul 21 17:02:55.943646 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:55.943620    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100e03cb_0ee7_4983_810d_0906466049c2.slice/crio-029f435f3a4ee53a1e841cb5282ed40e712bbe5402b09ff12dcb83aba8a8a547 WatchSource:0}: Error finding container 029f435f3a4ee53a1e841cb5282ed40e712bbe5402b09ff12dcb83aba8a8a547: Status 404 returned error can't find the container with id 029f435f3a4ee53a1e841cb5282ed40e712bbe5402b09ff12dcb83aba8a8a547
Jul 21 17:02:55.965314 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.965289    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gmllb"]
Jul 21 17:02:55.968506 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:55.968479    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24734282_d9c9_411e_b08b_748d83bc3fe4.slice/crio-f39f7ed3ae22da78c1701127f11b305c0c28bcbb7db785ff2d77cad341e4c85c WatchSource:0}: Error finding container f39f7ed3ae22da78c1701127f11b305c0c28bcbb7db785ff2d77cad341e4c85c: Status 404 returned error can't find the container with id f39f7ed3ae22da78c1701127f11b305c0c28bcbb7db785ff2d77cad341e4c85c
Jul 21 17:02:55.987093 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:55.987070    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-grf9q"]
Jul 21 17:02:55.989846 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:02:55.989822    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1868e074_fca4_47b5_a4e2_87e2eb73923d.slice/crio-fd27a5aa71009beda1633f4f02813eacf05dad202813ff3d91812ca812aae604 WatchSource:0}: Error finding container fd27a5aa71009beda1633f4f02813eacf05dad202813ff3d91812ca812aae604: Status 404 returned error can't find the container with id fd27a5aa71009beda1633f4f02813eacf05dad202813ff3d91812ca812aae604
Jul 21 17:02:56.512813 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.512783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:02:56.516603 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.516550    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5l8sr\""
Jul 21 17:02:56.517037 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.516854    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 17:02:56.782482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.782398    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmllb" event={"ID":"24734282-d9c9-411e-b08b-748d83bc3fe4","Type":"ContainerStarted","Data":"f39f7ed3ae22da78c1701127f11b305c0c28bcbb7db785ff2d77cad341e4c85c"}
Jul 21 17:02:56.784687 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.784654    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf" event={"ID":"100e03cb-0ee7-4983-810d-0906466049c2","Type":"ContainerStarted","Data":"1f97cb6dbaf0e7264b6ce740a8f807ea7f1342f976f955a7bf1a751f84238a21"}
Jul 21 17:02:56.784800 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.784707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf" event={"ID":"100e03cb-0ee7-4983-810d-0906466049c2","Type":"ContainerStarted","Data":"029f435f3a4ee53a1e841cb5282ed40e712bbe5402b09ff12dcb83aba8a8a547"}
Jul 21 17:02:56.785475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.785413    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:02:56.788003 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.787939    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-grf9q" event={"ID":"1868e074-fca4-47b5-a4e2-87e2eb73923d","Type":"ContainerStarted","Data":"fd27a5aa71009beda1633f4f02813eacf05dad202813ff3d91812ca812aae604"}
Jul 21 17:02:56.790061 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:56.789998    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pcgv" event={"ID":"9b75e216-b714-4036-8214-ca7adddb4839","Type":"ContainerStarted","Data":"268d1eccf1e1a537ef677a69410dd673939a75c7205be64ab5dcdff26ed77383"}
Jul 21 17:02:58.496219 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.496156    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf" podStartSLOduration=4.4961378 podStartE2EDuration="4.4961378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:56.81087373 +0000 UTC m=+56.775773338" watchObservedRunningTime="2026-07-21 17:02:58.4961378 +0000 UTC m=+58.461037403"
Jul 21 17:02:58.496748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.496613    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"]
Jul 21 17:02:58.498765 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.498738    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:02:58.501668 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.501643    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 21 17:02:58.501783 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.501689    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dtmdg\""
Jul 21 17:02:58.513949 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.513922    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"]
Jul 21 17:02:58.565187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.565143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086997e7-6677-46f5-be2b-b7852e9bb70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-68zmz\" (UID: \"086997e7-6677-46f5-be2b-b7852e9bb70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:02:58.666279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.666245    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086997e7-6677-46f5-be2b-b7852e9bb70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-68zmz\" (UID: \"086997e7-6677-46f5-be2b-b7852e9bb70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:02:58.666448 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:58.666383    2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 17:02:58.666511 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:02:58.666466    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086997e7-6677-46f5-be2b-b7852e9bb70a-tls-certificates podName:086997e7-6677-46f5-be2b-b7852e9bb70a nodeName:}" failed. No retries permitted until 2026-07-21 17:02:59.166445684 +0000 UTC m=+59.131345269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/086997e7-6677-46f5-be2b-b7852e9bb70a-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-68zmz" (UID: "086997e7-6677-46f5-be2b-b7852e9bb70a") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 21 17:02:58.798318 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.798231    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pcgv" event={"ID":"9b75e216-b714-4036-8214-ca7adddb4839","Type":"ContainerStarted","Data":"5f23c8b6c971976bb384570afb2e709f32bc361553df5a8ec6c706ad2bda5aaa"}
Jul 21 17:02:58.798318 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.798301    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pcgv" event={"ID":"9b75e216-b714-4036-8214-ca7adddb4839","Type":"ContainerStarted","Data":"a91e0a5af22bb2fc1a421c0cd99ff7eb9401a9b15932876770d7363dbfcbec89"}
Jul 21 17:02:58.824246 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:58.824200    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2pcgv" podStartSLOduration=1.859617009 podStartE2EDuration="3.824183543s" totalImagesPullingTime="1.964566534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:58.823324346 +0000 UTC m=+58.788223953" watchObservedRunningTime="2026-07-21 17:02:58.824183543 +0000 UTC m=+58.789083160"
Jul 21 17:02:59.171232 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:59.171189    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086997e7-6677-46f5-be2b-b7852e9bb70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-68zmz\" (UID: \"086997e7-6677-46f5-be2b-b7852e9bb70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:02:59.174363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:59.174332    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/086997e7-6677-46f5-be2b-b7852e9bb70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-68zmz\" (UID: \"086997e7-6677-46f5-be2b-b7852e9bb70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:02:59.411187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:59.411123    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:02:59.801061 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:02:59.801031    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:03:02.206929 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:02.206897    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"]
Jul 21 17:03:02.209920 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:02.209890    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086997e7_6677_46f5_be2b_b7852e9bb70a.slice/crio-3a6730d422fb03c63444cd8aefe5201c5550f880ea36fc891647294a301c0c49 WatchSource:0}: Error finding container 3a6730d422fb03c63444cd8aefe5201c5550f880ea36fc891647294a301c0c49: Status 404 returned error can't find the container with id 3a6730d422fb03c63444cd8aefe5201c5550f880ea36fc891647294a301c0c49
Jul 21 17:03:02.811576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:02.811516    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-grf9q" event={"ID":"1868e074-fca4-47b5-a4e2-87e2eb73923d","Type":"ContainerStarted","Data":"01fe53c5c20f10c01b1bf74ef63af07214d65489b104d48e793298d9d6db87c2"}
Jul 21 17:03:02.812702 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:02.812662    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz" event={"ID":"086997e7-6677-46f5-be2b-b7852e9bb70a","Type":"ContainerStarted","Data":"3a6730d422fb03c63444cd8aefe5201c5550f880ea36fc891647294a301c0c49"}
Jul 21 17:03:02.813956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:02.813937    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmllb" event={"ID":"24734282-d9c9-411e-b08b-748d83bc3fe4","Type":"ContainerStarted","Data":"dee6c1d7b985ea18b72f9a7807c572b334b22229ee6b3035b8465c7a4984a7f9"}
Jul 21 17:03:02.836216 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:02.836174    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-grf9q" podStartSLOduration=1.7437025240000001 podStartE2EDuration="7.836159649s" totalImagesPullingTime="6.092457125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:02.835122575 +0000 UTC m=+62.800022179" watchObservedRunningTime="2026-07-21 17:03:02.836159649 +0000 UTC m=+62.801059253"
Jul 21 17:03:03.741625 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:03.741598    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qczpg"
Jul 21 17:03:03.819782 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:03.819749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz" event={"ID":"086997e7-6677-46f5-be2b-b7852e9bb70a","Type":"ContainerStarted","Data":"dabc81ea1df6a5e3ba338a5ccbbb2473124c9bfbed5ab03e88ce7d394d37eb8c"}
Jul 21 17:03:03.820094 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:03.820047    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:03:03.825421 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:03.825399    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz"
Jul 21 17:03:03.842867 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:03.842822    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-68zmz" podStartSLOduration=4.35704524 podStartE2EDuration="5.842807469s" totalImagesPullingTime="1.485762229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:03.841131 +0000 UTC m=+63.806030608" watchObservedRunningTime="2026-07-21 17:03:03.842807469 +0000 UTC m=+63.807707070"
Jul 21 17:03:06.228125 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.228083    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:03:06.230930 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.230912    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 17:03:06.241167 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.241146    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c-metrics-certs\") pod \"network-metrics-daemon-wpvhl\" (UID: \"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c\") " pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:03:06.328899 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.328870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:03:06.331459 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.331442    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 17:03:06.341430 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.341408    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c24b7c8-f986-4410-bf06-5d31d80f43b6-original-pull-secret\") pod \"global-pull-secret-syncer-jjmrd\" (UID: \"0c24b7c8-f986-4410-bf06-5d31d80f43b6\") " pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:03:06.428250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.428221    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5l8sr\""
Jul 21 17:03:06.429416 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.429399    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:03:06.431702 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.431684    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 17:03:06.436533 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.436518    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpvhl"
Jul 21 17:03:06.441660 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.441646    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 17:03:06.452224 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.452200    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9gvd\" (UniqueName: \"kubernetes.io/projected/849ea74a-b773-4c55-85b1-a891929c6e25-kube-api-access-b9gvd\") pod \"network-check-target-d6h64\" (UID: \"849ea74a-b773-4c55-85b1-a891929c6e25\") " pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:03:06.561257 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.561225    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wpvhl"]
Jul 21 17:03:06.564274 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:06.564251    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5784d0_92b0_4cb4_ad11_e8df4cfeea5c.slice/crio-0e96d05b21e640024e3ce17c9d14ac9749ffd005556d7d716c03b82689d28c5d WatchSource:0}: Error finding container 0e96d05b21e640024e3ce17c9d14ac9749ffd005556d7d716c03b82689d28c5d: Status 404 returned error can't find the container with id 0e96d05b21e640024e3ce17c9d14ac9749ffd005556d7d716c03b82689d28c5d
Jul 21 17:03:06.618149 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.618127    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-59btl\""
Jul 21 17:03:06.620167 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.620143    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjmrd"
Jul 21 17:03:06.626599 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.626582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:03:06.745735 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.745648    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jjmrd"]
Jul 21 17:03:06.750295 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:06.750266    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c24b7c8_f986_4410_bf06_5d31d80f43b6.slice/crio-85540787c20286dc3fd7f9d9017c787baa99167bfe90eb9018db6158ce7a0e2a WatchSource:0}: Error finding container 85540787c20286dc3fd7f9d9017c787baa99167bfe90eb9018db6158ce7a0e2a: Status 404 returned error can't find the container with id 85540787c20286dc3fd7f9d9017c787baa99167bfe90eb9018db6158ce7a0e2a
Jul 21 17:03:06.765212 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.765189    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d6h64"]
Jul 21 17:03:06.767167 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:06.767139    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849ea74a_b773_4c55_85b1_a891929c6e25.slice/crio-cc0bc08fc33f6c678d5e770d85784d5aea40e9eb127620e47261f8a7fa742c96 WatchSource:0}: Error finding container cc0bc08fc33f6c678d5e770d85784d5aea40e9eb127620e47261f8a7fa742c96: Status 404 returned error can't find the container with id cc0bc08fc33f6c678d5e770d85784d5aea40e9eb127620e47261f8a7fa742c96
Jul 21 17:03:06.832118 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.832081    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wpvhl" event={"ID":"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c","Type":"ContainerStarted","Data":"0e96d05b21e640024e3ce17c9d14ac9749ffd005556d7d716c03b82689d28c5d"}
Jul 21 17:03:06.833721 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.833693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmllb" event={"ID":"24734282-d9c9-411e-b08b-748d83bc3fe4","Type":"ContainerStarted","Data":"ba51e8e32bd4e4672a193661f18b4bbd6670f9b5162fd5a38df3236d38abe16b"}
Jul 21 17:03:06.834860 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.834841    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jjmrd" event={"ID":"0c24b7c8-f986-4410-bf06-5d31d80f43b6","Type":"ContainerStarted","Data":"85540787c20286dc3fd7f9d9017c787baa99167bfe90eb9018db6158ce7a0e2a"}
Jul 21 17:03:06.835870 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:06.835848    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d6h64" event={"ID":"849ea74a-b773-4c55-85b1-a891929c6e25","Type":"ContainerStarted","Data":"cc0bc08fc33f6c678d5e770d85784d5aea40e9eb127620e47261f8a7fa742c96"}
Jul 21 17:03:08.850467 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:08.850428    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wpvhl" event={"ID":"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c","Type":"ContainerStarted","Data":"6d3ca997e67b86a97055c70b84182e69ef4b88434c5944b7be1808c6b78425f9"}
Jul 21 17:03:08.850937 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:08.850483    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wpvhl" event={"ID":"7a5784d0-92b0-4cb4-ad11-e8df4cfeea5c","Type":"ContainerStarted","Data":"48db05912314ad4e5277e5bc6d87547a21f8d497501e0f7368a18019582be4a8"}
Jul 21 17:03:08.874097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:08.874038    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wpvhl" podStartSLOduration=66.526155088 podStartE2EDuration="1m7.874016707s" totalImagesPullingTime="1.347861619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:08.872785644 +0000 UTC m=+68.837685248" watchObservedRunningTime="2026-07-21 17:03:08.874016707 +0000 UTC m=+68.838916313"
Jul 21 17:03:09.807123 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:09.807081    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2pcgv"
Jul 21 17:03:12.963414 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.963373    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-44b96"]
Jul 21 17:03:12.965957 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.965928    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:12.968621 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.968598    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 17:03:12.970975 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.970951    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 17:03:12.971716 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.971475    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 17:03:12.971716 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.971538    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 17:03:12.971716 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.971535    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 17:03:12.971716 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.971614    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 17:03:12.971947 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.971743    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fks9h\""
Jul 21 17:03:12.994820 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.994797    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"]
Jul 21 17:03:12.997094 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:12.997073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.001193 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.001170    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fxznw\""
Jul 21 17:03:13.001818 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.001622    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 21 17:03:13.001818 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.001640    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 21 17:03:13.001818 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.001720    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 21 17:03:13.014629 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.014607    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"]
Jul 21 17:03:13.082329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-textfile\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082340    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/90b3084e-d2d0-47f2-858e-64683244d001-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.082469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082372    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.082469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082430    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082451    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-root\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-tls\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082499    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-accelerators-collector-config\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082529    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4696\" (UniqueName: \"kubernetes.io/projected/90b3084e-d2d0-47f2-858e-64683244d001-kube-api-access-j4696\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.082719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082623    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.082719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082666    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-sys\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082693    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90b3084e-d2d0-47f2-858e-64683244d001-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.082933 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082721    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-metrics-client-ca\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082933 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082753    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-wtmp\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.082933 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082809    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.082933 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.082835    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfw28\" (UniqueName: \"kubernetes.io/projected/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-kube-api-access-wfw28\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.183627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-textfile\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.183627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/90b3084e-d2d0-47f2-858e-64683244d001-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.183832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.183832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.183832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-root\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.183832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183760    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-tls\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.183832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-accelerators-collector-config\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.183832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4696\" (UniqueName: \"kubernetes.io/projected/90b3084e-d2d0-47f2-858e-64683244d001-kube-api-access-j4696\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183851    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183883    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-sys\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183909    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90b3084e-d2d0-47f2-858e-64683244d001-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-metrics-client-ca\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183969    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-wtmp\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.183996    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-textfile\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184012    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfw28\" (UniqueName: \"kubernetes.io/projected/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-kube-api-access-wfw28\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184060    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/90b3084e-d2d0-47f2-858e-64683244d001-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:03:13.184149    2561 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-wtmp\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:03:13.184208    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-tls podName:90b3084e-d2d0-47f2-858e-64683244d001 nodeName:}" failed. No retries permitted until 2026-07-21 17:03:13.684189136 +0000 UTC m=+73.649088725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-t9rs5" (UID: "90b3084e-d2d0-47f2-858e-64683244d001") : secret "kube-state-metrics-tls" not found
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:03:13.184253    2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:03:13.184316    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-tls podName:85e079b2-abd1-4fdf-b22a-28f4e78c2a26 nodeName:}" failed. No retries permitted until 2026-07-21 17:03:13.68429829 +0000 UTC m=+73.649197887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-tls") pod "node-exporter-44b96" (UID: "85e079b2-abd1-4fdf-b22a-28f4e78c2a26") : secret "node-exporter-tls" not found
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184340    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-sys\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184536 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184485    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184543    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-root\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.184927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90b3084e-d2d0-47f2-858e-64683244d001-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.184927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.184793    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-accelerators-collector-config\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.185088 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.185017    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-metrics-client-ca\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.186990 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.186970    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.187177 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.187155    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.195576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.195530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4696\" (UniqueName: \"kubernetes.io/projected/90b3084e-d2d0-47f2-858e-64683244d001-kube-api-access-j4696\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.196462 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.196433    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfw28\" (UniqueName: \"kubernetes.io/projected/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-kube-api-access-wfw28\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.686875 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.686837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.687017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.686896    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-tls\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.689265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.689245    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85e079b2-abd1-4fdf-b22a-28f4e78c2a26-node-exporter-tls\") pod \"node-exporter-44b96\" (UID: \"85e079b2-abd1-4fdf-b22a-28f4e78c2a26\") " pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.689498 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.689477    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90b3084e-d2d0-47f2-858e-64683244d001-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-t9rs5\" (UID: \"90b3084e-d2d0-47f2-858e-64683244d001\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.876446 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.876415    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44b96"
Jul 21 17:03:13.880762 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.880733    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gmllb" event={"ID":"24734282-d9c9-411e-b08b-748d83bc3fe4","Type":"ContainerStarted","Data":"ec03174d7ec8e87f6348f0b15c8a63dbbb33636f4ea583d6fc98b6b91f7cd89c"}
Jul 21 17:03:13.882219 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.882194    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jjmrd" event={"ID":"0c24b7c8-f986-4410-bf06-5d31d80f43b6","Type":"ContainerStarted","Data":"e2a24220ab1aefcd5719b1824cb3d816e604bc19a5e01d8f84f022a97b16187d"}
Jul 21 17:03:13.883699 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.883672    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d6h64" event={"ID":"849ea74a-b773-4c55-85b1-a891929c6e25","Type":"ContainerStarted","Data":"e01f18c660c471dfee95d1caee1647c6f6065020c2b84c67e478b0aa8d564eb3"}
Jul 21 17:03:13.883799 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.883784    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:03:13.899169 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:13.899144    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e079b2_abd1_4fdf_b22a_28f4e78c2a26.slice/crio-b03c4ebbee46848ce25bee868fa61bc819f0883f2d729fa80fbd1e417d9c192c WatchSource:0}: Error finding container b03c4ebbee46848ce25bee868fa61bc819f0883f2d729fa80fbd1e417d9c192c: Status 404 returned error can't find the container with id b03c4ebbee46848ce25bee868fa61bc819f0883f2d729fa80fbd1e417d9c192c
Jul 21 17:03:13.900894 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.900792    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gmllb" podStartSLOduration=1.601754654 podStartE2EDuration="18.900771267s" totalImagesPullingTime="17.299016613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:13.898695823 +0000 UTC m=+73.863595426" watchObservedRunningTime="2026-07-21 17:03:13.900771267 +0000 UTC m=+73.865670875"
Jul 21 17:03:13.907440 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.907424    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"
Jul 21 17:03:13.920442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.920398    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d6h64" podStartSLOduration=66.295719805 podStartE2EDuration="1m12.9203797s" totalImagesPullingTime="6.624659895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:13.919855184 +0000 UTC m=+73.884754789" watchObservedRunningTime="2026-07-21 17:03:13.9203797 +0000 UTC m=+73.885279305"
Jul 21 17:03:13.939711 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:13.939655    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jjmrd" podStartSLOduration=66.282981298 podStartE2EDuration="1m12.939643211s" totalImagesPullingTime="6.656661913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:13.939407188 +0000 UTC m=+73.904306805" watchObservedRunningTime="2026-07-21 17:03:13.939643211 +0000 UTC m=+73.904542814"
Jul 21 17:03:14.062247 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:14.062163    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-t9rs5"]
Jul 21 17:03:14.065752 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:14.065728    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b3084e_d2d0_47f2_858e_64683244d001.slice/crio-7b48118accc6de7eadca95dbe42a7d9956a95455f49e43bb08c5c1b3b5c213af WatchSource:0}: Error finding container 7b48118accc6de7eadca95dbe42a7d9956a95455f49e43bb08c5c1b3b5c213af: Status 404 returned error can't find the container with id 7b48118accc6de7eadca95dbe42a7d9956a95455f49e43bb08c5c1b3b5c213af
Jul 21 17:03:14.887749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:14.887706    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44b96" event={"ID":"85e079b2-abd1-4fdf-b22a-28f4e78c2a26","Type":"ContainerStarted","Data":"b03c4ebbee46848ce25bee868fa61bc819f0883f2d729fa80fbd1e417d9c192c"}
Jul 21 17:03:14.888794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:14.888764    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5" event={"ID":"90b3084e-d2d0-47f2-858e-64683244d001","Type":"ContainerStarted","Data":"7b48118accc6de7eadca95dbe42a7d9956a95455f49e43bb08c5c1b3b5c213af"}
Jul 21 17:03:15.700825 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:15.700782    2561 patch_prober.go:28] interesting pod/image-registry-5b8bb8d44-wvlbf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 21 17:03:15.701200 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:15.700856    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf" podUID="100e03cb-0ee7-4983-810d-0906466049c2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 17:03:15.893385 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:15.893349    2561 generic.go:356] "Generic (PLEG): container finished" podID="85e079b2-abd1-4fdf-b22a-28f4e78c2a26" containerID="67ae4c5fddd8238310f2f002355d9f7e7aea08f5f3e92768bf79d9c55e3c3354" exitCode=0
Jul 21 17:03:15.893525 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:15.893423    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44b96" event={"ID":"85e079b2-abd1-4fdf-b22a-28f4e78c2a26","Type":"ContainerDied","Data":"67ae4c5fddd8238310f2f002355d9f7e7aea08f5f3e92768bf79d9c55e3c3354"}
Jul 21 17:03:16.898969 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.898925    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5" event={"ID":"90b3084e-d2d0-47f2-858e-64683244d001","Type":"ContainerStarted","Data":"5381f0be8377873d6144172ec388e291ce9f94cf4bc017b3e082cd644685d5f7"}
Jul 21 17:03:16.899381 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.899002    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5" event={"ID":"90b3084e-d2d0-47f2-858e-64683244d001","Type":"ContainerStarted","Data":"d6ec543a585452a0b61c3fafce9036d62918b40080667a665d4792ea237e2b8d"}
Jul 21 17:03:16.899381 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.899047    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5" event={"ID":"90b3084e-d2d0-47f2-858e-64683244d001","Type":"ContainerStarted","Data":"df06adafa5d31ea9b2497230268f44fcffa92ba26786e32163c61b458283663c"}
Jul 21 17:03:16.901024 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.901003    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44b96" event={"ID":"85e079b2-abd1-4fdf-b22a-28f4e78c2a26","Type":"ContainerStarted","Data":"e81b78bedeb3c2c762eb849ae683535bdb4ddfe17ac207677d8fbae5f071d14f"}
Jul 21 17:03:16.901119 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.901042    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44b96" event={"ID":"85e079b2-abd1-4fdf-b22a-28f4e78c2a26","Type":"ContainerStarted","Data":"d5f9bfff02e65bf70cd984a89e519bcff7459ebd4fde3781d93bf90237031f69"}
Jul 21 17:03:16.921097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.921048    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-t9rs5" podStartSLOduration=2.636692019 podStartE2EDuration="4.921034305s" totalImagesPullingTime="2.284342286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:16.920085205 +0000 UTC m=+76.884984823" watchObservedRunningTime="2026-07-21 17:03:16.921034305 +0000 UTC m=+76.885933984"
Jul 21 17:03:16.942125 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:16.942080    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-44b96" podStartSLOduration=3.060208624 podStartE2EDuration="4.942063478s" totalImagesPullingTime="1.810118672s" totalInitContainerRuntime="71.736182ms" isStatefulPod=true podCreationTimestamp="2026-07-21 17:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:16.940531119 +0000 UTC m=+76.905430723" watchObservedRunningTime="2026-07-21 17:03:16.942063478 +0000 UTC m=+76.906963082"
Jul 21 17:03:17.382609 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.382512    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-9f77db875-9mvj4"]
Jul 21 17:03:17.385680 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.385662    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.389166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.389128    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 21 17:03:17.389166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.389138    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 21 17:03:17.389349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.389219    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-60571i2650oij\""
Jul 21 17:03:17.389349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.389161    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 21 17:03:17.389349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.389153    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5f4kl\""
Jul 21 17:03:17.389454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.389361    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 21 17:03:17.397854 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.397833    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9f77db875-9mvj4"]
Jul 21 17:03:17.518820 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.518798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.518964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.518827    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-metrics-server-audit-profiles\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.518964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.518870    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-secret-metrics-server-tls\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.518964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.518937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-secret-metrics-server-client-certs\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.519117 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.518996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvqvn\" (UniqueName: \"kubernetes.io/projected/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-kube-api-access-qvqvn\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.519117 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.519021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-audit-log\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.519117 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.519040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-client-ca-bundle\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620365 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620323    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvqvn\" (UniqueName: \"kubernetes.io/projected/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-kube-api-access-qvqvn\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620502 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620412    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-audit-log\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620502 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-client-ca-bundle\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620578    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-metrics-server-audit-profiles\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-secret-metrics-server-tls\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620655    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-secret-metrics-server-client-certs\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.620854 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.620779    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-audit-log\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.621291 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.621264    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.621447 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.621420    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-metrics-server-audit-profiles\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.623162 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.623138    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-secret-metrics-server-tls\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.623253 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.623170    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-client-ca-bundle\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.623253 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.623183    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-secret-metrics-server-client-certs\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.629291 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.629269    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvqvn\" (UniqueName: \"kubernetes.io/projected/0b33d9fe-25d2-4825-af17-9e51cf0fe35a-kube-api-access-qvqvn\") pod \"metrics-server-9f77db875-9mvj4\" (UID: \"0b33d9fe-25d2-4825-af17-9e51cf0fe35a\") " pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.694641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.694609    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:17.832222 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.832189    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9f77db875-9mvj4"]
Jul 21 17:03:17.834828 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:17.834804    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b33d9fe_25d2_4825_af17_9e51cf0fe35a.slice/crio-34c3c4a6fd696801a13ae203a24e2aab218556a468fc4ba79910fa9d0a853171 WatchSource:0}: Error finding container 34c3c4a6fd696801a13ae203a24e2aab218556a468fc4ba79910fa9d0a853171: Status 404 returned error can't find the container with id 34c3c4a6fd696801a13ae203a24e2aab218556a468fc4ba79910fa9d0a853171
Jul 21 17:03:17.905209 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:17.905179    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4" event={"ID":"0b33d9fe-25d2-4825-af17-9e51cf0fe35a","Type":"ContainerStarted","Data":"34c3c4a6fd696801a13ae203a24e2aab218556a468fc4ba79910fa9d0a853171"}
Jul 21 17:03:18.198421 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.198390    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6df97fbddb-wts2s"]
Jul 21 17:03:18.202751 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.202728    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.204978 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.204954    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 21 17:03:18.205306 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.205286    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 21 17:03:18.205384 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.205318    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 21 17:03:18.205384 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.205331    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 21 17:03:18.205384 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.205288    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-l8vs6\""
Jul 21 17:03:18.205515 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.205286    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 21 17:03:18.209886 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.209866    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 21 17:03:18.214627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.214607    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6df97fbddb-wts2s"]
Jul 21 17:03:18.325872 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.325849    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7jjb\" (UniqueName: \"kubernetes.io/projected/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-kube-api-access-w7jjb\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.325877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.325895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-telemeter-client-tls\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.325928    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-metrics-client-ca\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.325968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-serving-certs-ca-bundle\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.325996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-secret-telemeter-client\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326163 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.326055    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-federate-client-tls\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.326163 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.326090    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.426734 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-telemeter-client-tls\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.426900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426754    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-metrics-client-ca\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.426900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-serving-certs-ca-bundle\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.426900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-secret-telemeter-client\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.426900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426859    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-federate-client-tls\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.426900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.427322 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426958    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7jjb\" (UniqueName: \"kubernetes.io/projected/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-kube-api-access-w7jjb\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.427322 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.426982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.427908 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.427516    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-metrics-client-ca\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.427908 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.427530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-serving-certs-ca-bundle\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.427908 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.427862    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.429832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.429811    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-secret-telemeter-client\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.430301 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.430270    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-telemeter-client-tls\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.430492 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.430475    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-federate-client-tls\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.430547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.430524    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.437464 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.437443    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7jjb\" (UniqueName: \"kubernetes.io/projected/a7100a61-1e0a-44b9-bc6c-46827d4e6ae8-kube-api-access-w7jjb\") pod \"telemeter-client-6df97fbddb-wts2s\" (UID: \"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8\") " pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.512924 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.512867    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s"
Jul 21 17:03:18.683693 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.683580    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6df97fbddb-wts2s"]
Jul 21 17:03:18.689623 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:18.689598    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7100a61_1e0a_44b9_bc6c_46827d4e6ae8.slice/crio-72ff813bd4963da225760474645b1543ae24de41d3d87ee7e284c56c6e326a3a WatchSource:0}: Error finding container 72ff813bd4963da225760474645b1543ae24de41d3d87ee7e284c56c6e326a3a: Status 404 returned error can't find the container with id 72ff813bd4963da225760474645b1543ae24de41d3d87ee7e284c56c6e326a3a
Jul 21 17:03:18.804071 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.803987    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b8bb8d44-wvlbf"
Jul 21 17:03:18.913698 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:18.913651    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s" event={"ID":"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8","Type":"ContainerStarted","Data":"72ff813bd4963da225760474645b1543ae24de41d3d87ee7e284c56c6e326a3a"}
Jul 21 17:03:19.517343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.517306    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7df6786cb4-4fqrd"]
Jul 21 17:03:19.520679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.520655    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.524062 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524039    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 17:03:19.524257 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524238    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7wcjw\""
Jul 21 17:03:19.524332 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524253    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 17:03:19.524387 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524342    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 17:03:19.524442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524258    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 17:03:19.524497 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524177    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 17:03:19.524547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524534    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 17:03:19.524631 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.524607    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 17:03:19.531691 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.531453    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 17:03:19.535210 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.535156    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7df6786cb4-4fqrd"]
Jul 21 17:03:19.636371 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636299    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-trusted-ca-bundle\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.636371 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-oauth-serving-cert\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.636371 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636359    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-oauth-config\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.636615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-serving-cert\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.636615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636418    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-config\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.636615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636438    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxjf\" (UniqueName: \"kubernetes.io/projected/8bf4d264-8e61-4f44-bc56-007c82cd235b-kube-api-access-hdxjf\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.636615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.636452    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-service-ca\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-oauth-config\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737632    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-serving-cert\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-config\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxjf\" (UniqueName: \"kubernetes.io/projected/8bf4d264-8e61-4f44-bc56-007c82cd235b-kube-api-access-hdxjf\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-service-ca\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-trusted-ca-bundle\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.737791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.737786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-oauth-serving-cert\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.738586 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.738528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-oauth-serving-cert\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.738853 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.738728    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-config\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.739207 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.739186    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-service-ca\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.739367 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.739338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-trusted-ca-bundle\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.740711 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.740686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-oauth-config\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.740877 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.740851    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-serving-cert\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.748417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.748380    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxjf\" (UniqueName: \"kubernetes.io/projected/8bf4d264-8e61-4f44-bc56-007c82cd235b-kube-api-access-hdxjf\") pod \"console-7df6786cb4-4fqrd\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") " pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.836004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.835977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:19.920885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.920850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4" event={"ID":"0b33d9fe-25d2-4825-af17-9e51cf0fe35a","Type":"ContainerStarted","Data":"f825dcce3af00b18834130e4405f054df7392dfa2bcf7272e82e285a967c60ad"}
Jul 21 17:03:19.978757 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.978681    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4" podStartSLOduration=1.503090856 podStartE2EDuration="2.978659472s" totalImagesPullingTime="1.475568616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:19.94352703 +0000 UTC m=+79.908426634" watchObservedRunningTime="2026-07-21 17:03:19.978659472 +0000 UTC m=+79.943559079"
Jul 21 17:03:19.979797 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:19.979773    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7df6786cb4-4fqrd"]
Jul 21 17:03:19.983514 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:19.983485    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf4d264_8e61_4f44_bc56_007c82cd235b.slice/crio-2056a0644a677bffd4e575f22ec1e13780f5cd029580d690b6c35c8c2cfef1e0 WatchSource:0}: Error finding container 2056a0644a677bffd4e575f22ec1e13780f5cd029580d690b6c35c8c2cfef1e0: Status 404 returned error can't find the container with id 2056a0644a677bffd4e575f22ec1e13780f5cd029580d690b6c35c8c2cfef1e0
Jul 21 17:03:20.927354 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:20.927161    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df6786cb4-4fqrd" event={"ID":"8bf4d264-8e61-4f44-bc56-007c82cd235b","Type":"ContainerStarted","Data":"2056a0644a677bffd4e575f22ec1e13780f5cd029580d690b6c35c8c2cfef1e0"}
Jul 21 17:03:21.936039 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:21.935997    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s" event={"ID":"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8","Type":"ContainerStarted","Data":"e8c76ba3c62012d978702caa1882d57e012ed8714ad61ef500e6efb704544302"}
Jul 21 17:03:22.941420 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:22.941382    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df6786cb4-4fqrd" event={"ID":"8bf4d264-8e61-4f44-bc56-007c82cd235b","Type":"ContainerStarted","Data":"d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd"}
Jul 21 17:03:22.965057 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:22.964989    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7df6786cb4-4fqrd" podStartSLOduration=1.167320376 podStartE2EDuration="3.964970178s" totalImagesPullingTime="2.797649802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:22.964843766 +0000 UTC m=+82.929743371" watchObservedRunningTime="2026-07-21 17:03:22.964970178 +0000 UTC m=+82.929869784"
Jul 21 17:03:23.947466 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:23.947427    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s" event={"ID":"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8","Type":"ContainerStarted","Data":"6dee4e03558d2915e62cca31ec0fc3f843710d6ad2c14f2867b18cb0ff0c2e9e"}
Jul 21 17:03:23.947870 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:23.947489    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s" event={"ID":"a7100a61-1e0a-44b9-bc6c-46827d4e6ae8","Type":"ContainerStarted","Data":"73ebae67943f6ca61d2dcb3475dc823e30ace26773b71e339d5edc57dc957ce7"}
Jul 21 17:03:23.972263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:23.972202    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6df97fbddb-wts2s" podStartSLOduration=1.3485370589999999 podStartE2EDuration="5.972187776s" totalImagesPullingTime="4.623650717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:23.970838807 +0000 UTC m=+83.935738410" watchObservedRunningTime="2026-07-21 17:03:23.972187776 +0000 UTC m=+83.937087380"
Jul 21 17:03:24.965807 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:24.965774    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7df6786cb4-4fqrd"]
Jul 21 17:03:25.009094 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.007307    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db79d7bdd-zj9b8"]
Jul 21 17:03:25.013094 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.013074    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.033734 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.033713    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db79d7bdd-zj9b8"]
Jul 21 17:03:25.184282 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-service-ca\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.184282 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184284    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-console-config\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.184472 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-serving-cert\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.184472 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184403    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-trusted-ca-bundle\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.184472 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184437    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-oauth-config\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.184472 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184466    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m5xs\" (UniqueName: \"kubernetes.io/projected/462a9dee-7471-483f-bdae-b1f635fff86c-kube-api-access-6m5xs\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.184670 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.184494    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-oauth-serving-cert\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285215 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-serving-cert\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285215 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285190    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-trusted-ca-bundle\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285216    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-oauth-config\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285233    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m5xs\" (UniqueName: \"kubernetes.io/projected/462a9dee-7471-483f-bdae-b1f635fff86c-kube-api-access-6m5xs\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-oauth-serving-cert\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-service-ca\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.285376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.285335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-console-config\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.286079 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.286053    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-service-ca\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.286200 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.286053    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-oauth-serving-cert\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.286200 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.286177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-console-config\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.286312 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.286212    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-trusted-ca-bundle\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.287656 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.287627    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-oauth-config\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.287993 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.287975    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-serving-cert\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.297736 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.297707    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m5xs\" (UniqueName: \"kubernetes.io/projected/462a9dee-7471-483f-bdae-b1f635fff86c-kube-api-access-6m5xs\") pod \"console-6db79d7bdd-zj9b8\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") " pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.321518 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.321499    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:25.463643 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.463607    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db79d7bdd-zj9b8"]
Jul 21 17:03:25.467201 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:03:25.467173    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462a9dee_7471_483f_bdae_b1f635fff86c.slice/crio-da6d90f7d141d749cd10305b25fad8ccf00fdc327cf8d927ec845b43710ef149 WatchSource:0}: Error finding container da6d90f7d141d749cd10305b25fad8ccf00fdc327cf8d927ec845b43710ef149: Status 404 returned error can't find the container with id da6d90f7d141d749cd10305b25fad8ccf00fdc327cf8d927ec845b43710ef149
Jul 21 17:03:25.956027 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.955991    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db79d7bdd-zj9b8" event={"ID":"462a9dee-7471-483f-bdae-b1f635fff86c","Type":"ContainerStarted","Data":"23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614"}
Jul 21 17:03:25.956188 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.956041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db79d7bdd-zj9b8" event={"ID":"462a9dee-7471-483f-bdae-b1f635fff86c","Type":"ContainerStarted","Data":"da6d90f7d141d749cd10305b25fad8ccf00fdc327cf8d927ec845b43710ef149"}
Jul 21 17:03:25.975615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:25.975569    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6db79d7bdd-zj9b8" podStartSLOduration=1.975536467 podStartE2EDuration="1.975536467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:25.974781819 +0000 UTC m=+85.939681426" watchObservedRunningTime="2026-07-21 17:03:25.975536467 +0000 UTC m=+85.940436071"
Jul 21 17:03:29.836639 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:29.836604    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:35.322422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:35.322381    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:35.322888 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:35.322460    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:35.327225 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:35.327201    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:35.992837 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:35.992808    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:03:37.694787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:37.694731    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:37.695181 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:37.694821    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:44.891645 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:44.891615    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d6h64"
Jul 21 17:03:49.986996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:49.986938    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7df6786cb4-4fqrd" podUID="8bf4d264-8e61-4f44-bc56-007c82cd235b" containerName="console" containerID="cri-o://d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd" gracePeriod=15
Jul 21 17:03:50.234359 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.234329    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:50.259700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.259638    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-config\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.259700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.259665    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-trusted-ca-bundle\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.259700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.259687    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdxjf\" (UniqueName: \"kubernetes.io/projected/8bf4d264-8e61-4f44-bc56-007c82cd235b-kube-api-access-hdxjf\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.259927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.259718    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-oauth-config\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.259927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.259906    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-oauth-serving-cert\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.260037 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.259961    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-serving-cert\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.260037 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260013    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-service-ca\") pod \"8bf4d264-8e61-4f44-bc56-007c82cd235b\" (UID: \"8bf4d264-8e61-4f44-bc56-007c82cd235b\") "
Jul 21 17:03:50.260142 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260087    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:03:50.260195 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260172    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-config" (OuterVolumeSpecName: "console-config") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:03:50.260285 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260268    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-config\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:50.260337 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260307    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-trusted-ca-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:50.260337 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260259    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:03:50.260475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.260457    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:03:50.262540 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.262512    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:03:50.262708 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.262585    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:03:50.262708 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.262629    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf4d264-8e61-4f44-bc56-007c82cd235b-kube-api-access-hdxjf" (OuterVolumeSpecName: "kube-api-access-hdxjf") pod "8bf4d264-8e61-4f44-bc56-007c82cd235b" (UID: "8bf4d264-8e61-4f44-bc56-007c82cd235b"). InnerVolumeSpecName "kube-api-access-hdxjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:03:50.360863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.360832    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-oauth-serving-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:50.360863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.360861    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-serving-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:50.360863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.360871    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4d264-8e61-4f44-bc56-007c82cd235b-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:50.361068 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.360880    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxjf\" (UniqueName: \"kubernetes.io/projected/8bf4d264-8e61-4f44-bc56-007c82cd235b-kube-api-access-hdxjf\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:50.361068 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:50.360891    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf4d264-8e61-4f44-bc56-007c82cd235b-console-oauth-config\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:03:51.041191 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.041151    2561 generic.go:356] "Generic (PLEG): container finished" podID="8bf4d264-8e61-4f44-bc56-007c82cd235b" containerID="d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd" exitCode=0
Jul 21 17:03:51.041598 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.041213    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df6786cb4-4fqrd"
Jul 21 17:03:51.041598 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.041229    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df6786cb4-4fqrd" event={"ID":"8bf4d264-8e61-4f44-bc56-007c82cd235b","Type":"ContainerDied","Data":"d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd"}
Jul 21 17:03:51.041598 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.041277    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df6786cb4-4fqrd" event={"ID":"8bf4d264-8e61-4f44-bc56-007c82cd235b","Type":"ContainerDied","Data":"2056a0644a677bffd4e575f22ec1e13780f5cd029580d690b6c35c8c2cfef1e0"}
Jul 21 17:03:51.041598 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.041299    2561 scope.go:117] "RemoveContainer" containerID="d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd"
Jul 21 17:03:51.049570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.049535    2561 scope.go:117] "RemoveContainer" containerID="d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd"
Jul 21 17:03:51.049878 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:03:51.049857    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd\": container with ID starting with d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd not found: ID does not exist" containerID="d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd"
Jul 21 17:03:51.049928 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.049886    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd"} err="failed to get container status \"d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd\": rpc error: code = NotFound desc = could not find container \"d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd\": container with ID starting with d969d0a94a9d37dc0c297088b5783c116321a0d84e2d9c61854bae2d91431abd not found: ID does not exist"
Jul 21 17:03:51.066849 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.066820    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7df6786cb4-4fqrd"]
Jul 21 17:03:51.074853 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:51.074830    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7df6786cb4-4fqrd"]
Jul 21 17:03:52.511903 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:52.511871    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf4d264-8e61-4f44-bc56-007c82cd235b" path="/var/lib/kubelet/pods/8bf4d264-8e61-4f44-bc56-007c82cd235b/volumes"
Jul 21 17:03:57.700081 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:57.700049    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:03:57.703872 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:03:57.703852    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-9f77db875-9mvj4"
Jul 21 17:04:47.623534 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.623502    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85df7b475f-rtx2m"]
Jul 21 17:04:47.623987 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.623766    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf4d264-8e61-4f44-bc56-007c82cd235b" containerName="console"
Jul 21 17:04:47.623987 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.623777    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf4d264-8e61-4f44-bc56-007c82cd235b" containerName="console"
Jul 21 17:04:47.623987 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.623829    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf4d264-8e61-4f44-bc56-007c82cd235b" containerName="console"
Jul 21 17:04:47.626670 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.626651    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.645989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.645964    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85df7b475f-rtx2m"]
Jul 21 17:04:47.775125 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775088    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-serving-cert\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.775125 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775124    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-config\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.775330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775142    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8c95\" (UniqueName: \"kubernetes.io/projected/18c91dbb-be61-4edf-b6f7-8a676e181a15-kube-api-access-h8c95\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.775330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-oauth-config\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.775330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775218    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-oauth-serving-cert\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.775330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775238    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-trusted-ca-bundle\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.775330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.775273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-service-ca\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876155    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-serving-cert\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876186    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-config\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8c95\" (UniqueName: \"kubernetes.io/projected/18c91dbb-be61-4edf-b6f7-8a676e181a15-kube-api-access-h8c95\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876227    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-oauth-config\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876361    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-oauth-serving-cert\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876409    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-trusted-ca-bundle\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876577 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876458    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-service-ca\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.876956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.876932    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-config\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.877098 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.877079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-oauth-serving-cert\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.877150 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.877079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-service-ca\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.877334 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.877315    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-trusted-ca-bundle\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.879304 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.879278    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-serving-cert\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.879416 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.879394    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-oauth-config\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.886491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.886469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8c95\" (UniqueName: \"kubernetes.io/projected/18c91dbb-be61-4edf-b6f7-8a676e181a15-kube-api-access-h8c95\") pod \"console-85df7b475f-rtx2m\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") " pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:47.935315 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:47.935293    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:48.063581 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:48.063528    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85df7b475f-rtx2m"]
Jul 21 17:04:48.067267 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:04:48.067242    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c91dbb_be61_4edf_b6f7_8a676e181a15.slice/crio-ecaf922b435f04e8e408437249c160a057638b800e816f4ebef08f01b2556786 WatchSource:0}: Error finding container ecaf922b435f04e8e408437249c160a057638b800e816f4ebef08f01b2556786: Status 404 returned error can't find the container with id ecaf922b435f04e8e408437249c160a057638b800e816f4ebef08f01b2556786
Jul 21 17:04:48.236002 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:48.235964    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85df7b475f-rtx2m" event={"ID":"18c91dbb-be61-4edf-b6f7-8a676e181a15","Type":"ContainerStarted","Data":"0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836"}
Jul 21 17:04:48.236180 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:48.236014    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85df7b475f-rtx2m" event={"ID":"18c91dbb-be61-4edf-b6f7-8a676e181a15","Type":"ContainerStarted","Data":"ecaf922b435f04e8e408437249c160a057638b800e816f4ebef08f01b2556786"}
Jul 21 17:04:48.257107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:48.257035    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85df7b475f-rtx2m" podStartSLOduration=1.256977288 podStartE2EDuration="1.256977288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:04:48.256860897 +0000 UTC m=+168.221760502" watchObservedRunningTime="2026-07-21 17:04:48.256977288 +0000 UTC m=+168.221876892"
Jul 21 17:04:57.935622 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:57.935503    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:57.935622 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:57.935577    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:57.940397 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:57.940377    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:58.272996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:58.272900    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:04:58.324472 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:04:58.324442    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db79d7bdd-zj9b8"]
Jul 21 17:05:23.350665 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.350629    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6db79d7bdd-zj9b8" podUID="462a9dee-7471-483f-bdae-b1f635fff86c" containerName="console" containerID="cri-o://23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614" gracePeriod=15
Jul 21 17:05:23.588276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.588253    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:05:23.632415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.632354    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-service-ca\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.632415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.632383    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-trusted-ca-bundle\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.632576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.632416    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-serving-cert\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.632576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.632433    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-oauth-config\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.632576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.632450    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-oauth-serving-cert\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.633169 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.632949    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m5xs\" (UniqueName: \"kubernetes.io/projected/462a9dee-7471-483f-bdae-b1f635fff86c-kube-api-access-6m5xs\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.633169 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.633029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-console-config\") pod \"462a9dee-7471-483f-bdae-b1f635fff86c\" (UID: \"462a9dee-7471-483f-bdae-b1f635fff86c\") "
Jul 21 17:05:23.633169 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.633072    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:23.633169 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.633098    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-service-ca" (OuterVolumeSpecName: "service-ca") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:23.633429 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.633320    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:23.633830 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.633785    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-console-config" (OuterVolumeSpecName: "console-config") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:05:23.635664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.634811    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-oauth-serving-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:23.635664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.634841    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-console-config\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:23.635664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.635000    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:23.635664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.635024    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462a9dee-7471-483f-bdae-b1f635fff86c-trusted-ca-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:23.635664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.635405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:05:23.635942 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.635687    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:05:23.639214 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.639190    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/462a9dee-7471-483f-bdae-b1f635fff86c-kube-api-access-6m5xs" (OuterVolumeSpecName: "kube-api-access-6m5xs") pod "462a9dee-7471-483f-bdae-b1f635fff86c" (UID: "462a9dee-7471-483f-bdae-b1f635fff86c"). InnerVolumeSpecName "kube-api-access-6m5xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:05:23.735860 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.735842    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-serving-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:23.735860 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.735861    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/462a9dee-7471-483f-bdae-b1f635fff86c-console-oauth-config\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:23.735860 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:23.735870    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m5xs\" (UniqueName: \"kubernetes.io/projected/462a9dee-7471-483f-bdae-b1f635fff86c-kube-api-access-6m5xs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:05:24.360077 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.360046    2561 generic.go:356] "Generic (PLEG): container finished" podID="462a9dee-7471-483f-bdae-b1f635fff86c" containerID="23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614" exitCode=0
Jul 21 17:05:24.360454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.360115    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db79d7bdd-zj9b8"
Jul 21 17:05:24.360454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.360132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db79d7bdd-zj9b8" event={"ID":"462a9dee-7471-483f-bdae-b1f635fff86c","Type":"ContainerDied","Data":"23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614"}
Jul 21 17:05:24.360454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.360218    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db79d7bdd-zj9b8" event={"ID":"462a9dee-7471-483f-bdae-b1f635fff86c","Type":"ContainerDied","Data":"da6d90f7d141d749cd10305b25fad8ccf00fdc327cf8d927ec845b43710ef149"}
Jul 21 17:05:24.360454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.360241    2561 scope.go:117] "RemoveContainer" containerID="23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614"
Jul 21 17:05:24.368758 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.368740    2561 scope.go:117] "RemoveContainer" containerID="23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614"
Jul 21 17:05:24.369024 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:05:24.369002    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614\": container with ID starting with 23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614 not found: ID does not exist" containerID="23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614"
Jul 21 17:05:24.369091 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.369036    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614"} err="failed to get container status \"23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614\": rpc error: code = NotFound desc = could not find container \"23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614\": container with ID starting with 23f7dcf52779bc703ec58a0a46569fccdf3921e3e1bf9b8701643e535ef8f614 not found: ID does not exist"
Jul 21 17:05:24.386106 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.386085    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db79d7bdd-zj9b8"]
Jul 21 17:05:24.388303 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.388285    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db79d7bdd-zj9b8"]
Jul 21 17:05:24.511787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:24.511750    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="462a9dee-7471-483f-bdae-b1f635fff86c" path="/var/lib/kubelet/pods/462a9dee-7471-483f-bdae-b1f635fff86c/volumes"
Jul 21 17:05:42.568864 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.568830    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"]
Jul 21 17:05:42.569269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.569089    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462a9dee-7471-483f-bdae-b1f635fff86c" containerName="console"
Jul 21 17:05:42.569269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.569100    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="462a9dee-7471-483f-bdae-b1f635fff86c" containerName="console"
Jul 21 17:05:42.569269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.569161    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="462a9dee-7471-483f-bdae-b1f635fff86c" containerName="console"
Jul 21 17:05:42.573374 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.573356    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.577687 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.577666    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 17:05:42.577793 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.577739    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 17:05:42.577793 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.577769    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p9j9s\""
Jul 21 17:05:42.583304 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.583284    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"]
Jul 21 17:05:42.667992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.667957    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg6bv\" (UniqueName: \"kubernetes.io/projected/1c7fe140-988c-4d6a-9833-d38336dab181-kube-api-access-fg6bv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.668129 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.668009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.668129 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.668093    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.769107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.769076    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg6bv\" (UniqueName: \"kubernetes.io/projected/1c7fe140-988c-4d6a-9833-d38336dab181-kube-api-access-fg6bv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.769227 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.769123    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.769227 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.769156    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.769451 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.769437    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.769504 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.769487    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.778605 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.778585    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg6bv\" (UniqueName: \"kubernetes.io/projected/1c7fe140-988c-4d6a-9833-d38336dab181-kube-api-access-fg6bv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:42.881931 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:42.881880    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:05:43.018428 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:43.018404    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"]
Jul 21 17:05:43.020943 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:05:43.020915    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7fe140_988c_4d6a_9833_d38336dab181.slice/crio-59f849e20701f8bd1d8eaf60537f4500ace3cb1b0ac048660a6660cb97565392 WatchSource:0}: Error finding container 59f849e20701f8bd1d8eaf60537f4500ace3cb1b0ac048660a6660cb97565392: Status 404 returned error can't find the container with id 59f849e20701f8bd1d8eaf60537f4500ace3cb1b0ac048660a6660cb97565392
Jul 21 17:05:43.427339 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:43.427302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn" event={"ID":"1c7fe140-988c-4d6a-9833-d38336dab181","Type":"ContainerStarted","Data":"59f849e20701f8bd1d8eaf60537f4500ace3cb1b0ac048660a6660cb97565392"}
Jul 21 17:05:50.456390 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:50.456358    2561 generic.go:356] "Generic (PLEG): container finished" podID="1c7fe140-988c-4d6a-9833-d38336dab181" containerID="341544a7d60cbf5cdf3a1d524c7ccc526a70f417d0a431fc3027f4b68a67cc59" exitCode=0
Jul 21 17:05:50.456767 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:50.456407    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn" event={"ID":"1c7fe140-988c-4d6a-9833-d38336dab181","Type":"ContainerDied","Data":"341544a7d60cbf5cdf3a1d524c7ccc526a70f417d0a431fc3027f4b68a67cc59"}
Jul 21 17:05:53.469886 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:53.469848    2561 generic.go:356] "Generic (PLEG): container finished" podID="1c7fe140-988c-4d6a-9833-d38336dab181" containerID="3b0cf2749ada23a49b42b8bb5923cdbb0091556806ca0b2f7e1f1e64e61ac4d1" exitCode=0
Jul 21 17:05:53.470275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:05:53.469933    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn" event={"ID":"1c7fe140-988c-4d6a-9833-d38336dab181","Type":"ContainerDied","Data":"3b0cf2749ada23a49b42b8bb5923cdbb0091556806ca0b2f7e1f1e64e61ac4d1"}
Jul 21 17:06:01.505741 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:01.505705    2561 generic.go:356] "Generic (PLEG): container finished" podID="1c7fe140-988c-4d6a-9833-d38336dab181" containerID="78a59ec21604d7be8c3bc9eb81f991b987d94c280a1ac372662d5a399174a561" exitCode=0
Jul 21 17:06:01.506147 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:01.505798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn" event={"ID":"1c7fe140-988c-4d6a-9833-d38336dab181","Type":"ContainerDied","Data":"78a59ec21604d7be8c3bc9eb81f991b987d94c280a1ac372662d5a399174a561"}
Jul 21 17:06:02.630067 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.630032    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:06:02.733782 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.733747    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-util\") pod \"1c7fe140-988c-4d6a-9833-d38336dab181\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") "
Jul 21 17:06:02.733984 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.733830    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg6bv\" (UniqueName: \"kubernetes.io/projected/1c7fe140-988c-4d6a-9833-d38336dab181-kube-api-access-fg6bv\") pod \"1c7fe140-988c-4d6a-9833-d38336dab181\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") "
Jul 21 17:06:02.733984 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.733862    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-bundle\") pod \"1c7fe140-988c-4d6a-9833-d38336dab181\" (UID: \"1c7fe140-988c-4d6a-9833-d38336dab181\") "
Jul 21 17:06:02.734507 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.734485    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-bundle" (OuterVolumeSpecName: "bundle") pod "1c7fe140-988c-4d6a-9833-d38336dab181" (UID: "1c7fe140-988c-4d6a-9833-d38336dab181"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:06:02.736190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.736166    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7fe140-988c-4d6a-9833-d38336dab181-kube-api-access-fg6bv" (OuterVolumeSpecName: "kube-api-access-fg6bv") pod "1c7fe140-988c-4d6a-9833-d38336dab181" (UID: "1c7fe140-988c-4d6a-9833-d38336dab181"). InnerVolumeSpecName "kube-api-access-fg6bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:06:02.739002 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.738976    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-util" (OuterVolumeSpecName: "util") pod "1c7fe140-988c-4d6a-9833-d38336dab181" (UID: "1c7fe140-988c-4d6a-9833-d38336dab181"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:06:02.834794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.834695    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:02.834794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.834731    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg6bv\" (UniqueName: \"kubernetes.io/projected/1c7fe140-988c-4d6a-9833-d38336dab181-kube-api-access-fg6bv\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:02.834794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:02.834745    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c7fe140-988c-4d6a-9833-d38336dab181-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:03.513945 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:03.513910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn" event={"ID":"1c7fe140-988c-4d6a-9833-d38336dab181","Type":"ContainerDied","Data":"59f849e20701f8bd1d8eaf60537f4500ace3cb1b0ac048660a6660cb97565392"}
Jul 21 17:06:03.514096 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:03.513957    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59f849e20701f8bd1d8eaf60537f4500ace3cb1b0ac048660a6660cb97565392"
Jul 21 17:06:03.514096 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:03.513921    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vqcdn"
Jul 21 17:06:12.280577 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280522    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"]
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280887    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="extract"
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280904    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="extract"
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280926    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="pull"
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280933    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="pull"
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280952    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="util"
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.280961    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="util"
Jul 21 17:06:12.281130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.281027    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7fe140-988c-4d6a-9833-d38336dab181" containerName="extract"
Jul 21 17:06:12.284408 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.284387    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.287348 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.287328    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 21 17:06:12.287704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.287689    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lqmsj\""
Jul 21 17:06:12.290043 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.290026    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 21 17:06:12.298668 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.298648    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"]
Jul 21 17:06:12.412935 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.412903    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66c2a37d-e093-432a-aa71-a67a01873561-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hxlq5\" (UID: \"66c2a37d-e093-432a-aa71-a67a01873561\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.413105 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.412941    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kqk8\" (UniqueName: \"kubernetes.io/projected/66c2a37d-e093-432a-aa71-a67a01873561-kube-api-access-8kqk8\") pod \"cert-manager-operator-controller-manager-55c444bf86-hxlq5\" (UID: \"66c2a37d-e093-432a-aa71-a67a01873561\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.513552 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.513525    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66c2a37d-e093-432a-aa71-a67a01873561-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hxlq5\" (UID: \"66c2a37d-e093-432a-aa71-a67a01873561\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.513708 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.513574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kqk8\" (UniqueName: \"kubernetes.io/projected/66c2a37d-e093-432a-aa71-a67a01873561-kube-api-access-8kqk8\") pod \"cert-manager-operator-controller-manager-55c444bf86-hxlq5\" (UID: \"66c2a37d-e093-432a-aa71-a67a01873561\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.513875 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.513857    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66c2a37d-e093-432a-aa71-a67a01873561-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hxlq5\" (UID: \"66c2a37d-e093-432a-aa71-a67a01873561\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.523793 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.523774    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kqk8\" (UniqueName: \"kubernetes.io/projected/66c2a37d-e093-432a-aa71-a67a01873561-kube-api-access-8kqk8\") pod \"cert-manager-operator-controller-manager-55c444bf86-hxlq5\" (UID: \"66c2a37d-e093-432a-aa71-a67a01873561\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.593680 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.593598    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"
Jul 21 17:06:12.759892 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:12.759788    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5"]
Jul 21 17:06:12.762396 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:12.762370    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c2a37d_e093_432a_aa71_a67a01873561.slice/crio-ed21b34a5d837c0138539a8bc0d29c5a33f0d77bf1839d94bcda43c0cf07b96e WatchSource:0}: Error finding container ed21b34a5d837c0138539a8bc0d29c5a33f0d77bf1839d94bcda43c0cf07b96e: Status 404 returned error can't find the container with id ed21b34a5d837c0138539a8bc0d29c5a33f0d77bf1839d94bcda43c0cf07b96e
Jul 21 17:06:13.549267 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:13.549221    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5" event={"ID":"66c2a37d-e093-432a-aa71-a67a01873561","Type":"ContainerStarted","Data":"ed21b34a5d837c0138539a8bc0d29c5a33f0d77bf1839d94bcda43c0cf07b96e"}
Jul 21 17:06:15.558405 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:15.558370    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5" event={"ID":"66c2a37d-e093-432a-aa71-a67a01873561","Type":"ContainerStarted","Data":"971a6647e41c7aafc8281f594bc5d6eed2aacbfce8d3967d2d20f04a989347b4"}
Jul 21 17:06:15.601252 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:15.601198    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hxlq5" podStartSLOduration=1.721046679 podStartE2EDuration="3.601180439s" totalImagesPullingTime="1.88013376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:15.597887457 +0000 UTC m=+255.562787060" watchObservedRunningTime="2026-07-21 17:06:15.601180439 +0000 UTC m=+255.566080043"
Jul 21 17:06:16.512721 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.512689    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"]
Jul 21 17:06:16.515283 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.515262    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.518970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.518946    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p9j9s\""
Jul 21 17:06:16.519044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.518951    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 17:06:16.519551 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.519532    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 17:06:16.545223 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.545198    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"]
Jul 21 17:06:16.650084 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.650045    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.650444 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.650097    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.650444 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.650130    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4mdf\" (UniqueName: \"kubernetes.io/projected/62512d86-551d-4564-b398-048e978293a5-kube-api-access-v4mdf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.751475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.751440    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.751475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.751480    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.751699 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.751508    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4mdf\" (UniqueName: \"kubernetes.io/projected/62512d86-551d-4564-b398-048e978293a5-kube-api-access-v4mdf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.751863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.751843    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.751974 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.751955    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.768532 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.768461    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4mdf\" (UniqueName: \"kubernetes.io/projected/62512d86-551d-4564-b398-048e978293a5-kube-api-access-v4mdf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.824426 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.824391    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:16.962247 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:16.962223    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"]
Jul 21 17:06:16.964046 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:16.964016    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62512d86_551d_4564_b398_048e978293a5.slice/crio-ee89936be9a41566fefe7c83119cfbab31e1475ce52b384e95734ea506860d4e WatchSource:0}: Error finding container ee89936be9a41566fefe7c83119cfbab31e1475ce52b384e95734ea506860d4e: Status 404 returned error can't find the container with id ee89936be9a41566fefe7c83119cfbab31e1475ce52b384e95734ea506860d4e
Jul 21 17:06:17.568785 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:17.568752    2561 generic.go:356] "Generic (PLEG): container finished" podID="62512d86-551d-4564-b398-048e978293a5" containerID="1924ed8c913b15b0f0d8fc41bfdd4a033c49b96b80b428a800a4d029946046ce" exitCode=0
Jul 21 17:06:17.569056 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:17.568844    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n" event={"ID":"62512d86-551d-4564-b398-048e978293a5","Type":"ContainerDied","Data":"1924ed8c913b15b0f0d8fc41bfdd4a033c49b96b80b428a800a4d029946046ce"}
Jul 21 17:06:17.569056 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:17.568900    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n" event={"ID":"62512d86-551d-4564-b398-048e978293a5","Type":"ContainerStarted","Data":"ee89936be9a41566fefe7c83119cfbab31e1475ce52b384e95734ea506860d4e"}
Jul 21 17:06:19.330348 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.330319    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"]
Jul 21 17:06:19.336206 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.336178    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.349093 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.349072    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5hm6h\""
Jul 21 17:06:19.353194 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.353174    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 17:06:19.353324 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.353184    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 17:06:19.371471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.371426    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"]
Jul 21 17:06:19.477970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.477941    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c7788cd-103d-48a4-8f2d-37efaadfef3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f9g2z\" (UID: \"8c7788cd-103d-48a4-8f2d-37efaadfef3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.478144 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.477993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6hhd\" (UniqueName: \"kubernetes.io/projected/8c7788cd-103d-48a4-8f2d-37efaadfef3d-kube-api-access-p6hhd\") pod \"cert-manager-cainjector-559cdd8c87-f9g2z\" (UID: \"8c7788cd-103d-48a4-8f2d-37efaadfef3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.578506 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.578474    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hhd\" (UniqueName: \"kubernetes.io/projected/8c7788cd-103d-48a4-8f2d-37efaadfef3d-kube-api-access-p6hhd\") pod \"cert-manager-cainjector-559cdd8c87-f9g2z\" (UID: \"8c7788cd-103d-48a4-8f2d-37efaadfef3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.578707 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.578606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c7788cd-103d-48a4-8f2d-37efaadfef3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f9g2z\" (UID: \"8c7788cd-103d-48a4-8f2d-37efaadfef3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.588608 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.588530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c7788cd-103d-48a4-8f2d-37efaadfef3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f9g2z\" (UID: \"8c7788cd-103d-48a4-8f2d-37efaadfef3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.588608 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.588591    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6hhd\" (UniqueName: \"kubernetes.io/projected/8c7788cd-103d-48a4-8f2d-37efaadfef3d-kube-api-access-p6hhd\") pod \"cert-manager-cainjector-559cdd8c87-f9g2z\" (UID: \"8c7788cd-103d-48a4-8f2d-37efaadfef3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.658302 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.658266    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"
Jul 21 17:06:19.949790 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:19.949769    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z"]
Jul 21 17:06:19.952293 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:19.952270    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c7788cd_103d_48a4_8f2d_37efaadfef3d.slice/crio-82975b5999c12dbccc543450d388c194b299ec70a2a528c3de0a3b26a12d4a1a WatchSource:0}: Error finding container 82975b5999c12dbccc543450d388c194b299ec70a2a528c3de0a3b26a12d4a1a: Status 404 returned error can't find the container with id 82975b5999c12dbccc543450d388c194b299ec70a2a528c3de0a3b26a12d4a1a
Jul 21 17:06:20.585041 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:20.585011    2561 generic.go:356] "Generic (PLEG): container finished" podID="62512d86-551d-4564-b398-048e978293a5" containerID="dfc13187df9cf617ef56e829a75a50db9420e2e817eb64698dd58da6c5f9a3e8" exitCode=0
Jul 21 17:06:20.585490 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:20.585085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n" event={"ID":"62512d86-551d-4564-b398-048e978293a5","Type":"ContainerDied","Data":"dfc13187df9cf617ef56e829a75a50db9420e2e817eb64698dd58da6c5f9a3e8"}
Jul 21 17:06:20.586276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:20.586243    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z" event={"ID":"8c7788cd-103d-48a4-8f2d-37efaadfef3d","Type":"ContainerStarted","Data":"82975b5999c12dbccc543450d388c194b299ec70a2a528c3de0a3b26a12d4a1a"}
Jul 21 17:06:21.592669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:21.592630    2561 generic.go:356] "Generic (PLEG): container finished" podID="62512d86-551d-4564-b398-048e978293a5" containerID="a43045d79ae66222530f82c773508f7af9cc4d15c775b1cf59ae3acc5bb00380" exitCode=0
Jul 21 17:06:21.593125 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:21.592711    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n" event={"ID":"62512d86-551d-4564-b398-048e978293a5","Type":"ContainerDied","Data":"a43045d79ae66222530f82c773508f7af9cc4d15c775b1cf59ae3acc5bb00380"}
Jul 21 17:06:22.721001 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.720926    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:22.805049 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.805018    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-util\") pod \"62512d86-551d-4564-b398-048e978293a5\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") "
Jul 21 17:06:22.805217 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.805080    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-bundle\") pod \"62512d86-551d-4564-b398-048e978293a5\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") "
Jul 21 17:06:22.805217 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.805159    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4mdf\" (UniqueName: \"kubernetes.io/projected/62512d86-551d-4564-b398-048e978293a5-kube-api-access-v4mdf\") pod \"62512d86-551d-4564-b398-048e978293a5\" (UID: \"62512d86-551d-4564-b398-048e978293a5\") "
Jul 21 17:06:22.805417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.805393    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-bundle" (OuterVolumeSpecName: "bundle") pod "62512d86-551d-4564-b398-048e978293a5" (UID: "62512d86-551d-4564-b398-048e978293a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:06:22.807325 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.807302    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62512d86-551d-4564-b398-048e978293a5-kube-api-access-v4mdf" (OuterVolumeSpecName: "kube-api-access-v4mdf") pod "62512d86-551d-4564-b398-048e978293a5" (UID: "62512d86-551d-4564-b398-048e978293a5"). InnerVolumeSpecName "kube-api-access-v4mdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:06:22.809216 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.809177    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-util" (OuterVolumeSpecName: "util") pod "62512d86-551d-4564-b398-048e978293a5" (UID: "62512d86-551d-4564-b398-048e978293a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:06:22.906342 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.906307    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4mdf\" (UniqueName: \"kubernetes.io/projected/62512d86-551d-4564-b398-048e978293a5-kube-api-access-v4mdf\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:22.906342 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.906333    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:22.906342 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:22.906342    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/62512d86-551d-4564-b398-048e978293a5-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:23.601869 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.601843    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n"
Jul 21 17:06:23.602077 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.601842    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn9c4n" event={"ID":"62512d86-551d-4564-b398-048e978293a5","Type":"ContainerDied","Data":"ee89936be9a41566fefe7c83119cfbab31e1475ce52b384e95734ea506860d4e"}
Jul 21 17:06:23.602077 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.601969    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee89936be9a41566fefe7c83119cfbab31e1475ce52b384e95734ea506860d4e"
Jul 21 17:06:23.603514 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.603483    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z" event={"ID":"8c7788cd-103d-48a4-8f2d-37efaadfef3d","Type":"ContainerStarted","Data":"298ea9051072ecd25f04dc4363d6f9dc64a45ceba7578bff3c68590e222fe87f"}
Jul 21 17:06:23.629578 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.629503    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9g2z" podStartSLOduration=2.006161491 podStartE2EDuration="4.629485535s" totalImagesPullingTime="2.623324044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:23.62882423 +0000 UTC m=+263.593723833" watchObservedRunningTime="2026-07-21 17:06:23.629485535 +0000 UTC m=+263.594385140"
Jul 21 17:06:23.863372 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863286    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gts8n"]
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863668    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="pull"
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863687    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="pull"
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863718    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="util"
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863725    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="util"
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863733    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="extract"
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863741    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="extract"
Jul 21 17:06:23.863814 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.863804    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="62512d86-551d-4564-b398-048e978293a5" containerName="extract"
Jul 21 17:06:23.867313 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.867293    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:23.870415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.870397    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lbgms\""
Jul 21 17:06:23.878392 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:23.878368    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gts8n"]
Jul 21 17:06:24.013500 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.013461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wthj\" (UniqueName: \"kubernetes.io/projected/73d1f196-d2df-4a0a-941f-95c61ecd6ae1-kube-api-access-5wthj\") pod \"cert-manager-webhook-6486ffd997-gts8n\" (UID: \"73d1f196-d2df-4a0a-941f-95c61ecd6ae1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.013697 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.013508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73d1f196-d2df-4a0a-941f-95c61ecd6ae1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gts8n\" (UID: \"73d1f196-d2df-4a0a-941f-95c61ecd6ae1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.114119 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.114032    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wthj\" (UniqueName: \"kubernetes.io/projected/73d1f196-d2df-4a0a-941f-95c61ecd6ae1-kube-api-access-5wthj\") pod \"cert-manager-webhook-6486ffd997-gts8n\" (UID: \"73d1f196-d2df-4a0a-941f-95c61ecd6ae1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.114119 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.114078    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73d1f196-d2df-4a0a-941f-95c61ecd6ae1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gts8n\" (UID: \"73d1f196-d2df-4a0a-941f-95c61ecd6ae1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.123871 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.123847    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73d1f196-d2df-4a0a-941f-95c61ecd6ae1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gts8n\" (UID: \"73d1f196-d2df-4a0a-941f-95c61ecd6ae1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.123994 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.123918    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wthj\" (UniqueName: \"kubernetes.io/projected/73d1f196-d2df-4a0a-941f-95c61ecd6ae1-kube-api-access-5wthj\") pod \"cert-manager-webhook-6486ffd997-gts8n\" (UID: \"73d1f196-d2df-4a0a-941f-95c61ecd6ae1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.175887 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.175860    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:24.305008 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.304981    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gts8n"]
Jul 21 17:06:24.308208 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:24.308183    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73d1f196_d2df_4a0a_941f_95c61ecd6ae1.slice/crio-b1477d49deff618926840bf77337db7c6fd4b208ecd8fffb640d2a50268c05bd WatchSource:0}: Error finding container b1477d49deff618926840bf77337db7c6fd4b208ecd8fffb640d2a50268c05bd: Status 404 returned error can't find the container with id b1477d49deff618926840bf77337db7c6fd4b208ecd8fffb640d2a50268c05bd
Jul 21 17:06:24.609127 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.609090    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n" event={"ID":"73d1f196-d2df-4a0a-941f-95c61ecd6ae1","Type":"ContainerStarted","Data":"36d0b5e7e409fa6a14fd6baf3189c1d44a717cbc34513b592e3c66fabaf088df"}
Jul 21 17:06:24.609325 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.609144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n" event={"ID":"73d1f196-d2df-4a0a-941f-95c61ecd6ae1","Type":"ContainerStarted","Data":"b1477d49deff618926840bf77337db7c6fd4b208ecd8fffb640d2a50268c05bd"}
Jul 21 17:06:24.629462 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:24.629371    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n" podStartSLOduration=1.629356955 podStartE2EDuration="1.629356955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:24.628675046 +0000 UTC m=+264.593574644" watchObservedRunningTime="2026-07-21 17:06:24.629356955 +0000 UTC m=+264.594256550"
Jul 21 17:06:25.613173 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:25.613141    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:31.619475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:31.619442    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gts8n"
Jul 21 17:06:32.875135 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.875100    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"]
Jul 21 17:06:32.877226 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.877211    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:32.879834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.879813    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 17:06:32.880704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.880689    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4hwvb\""
Jul 21 17:06:32.880777 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.880705    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 17:06:32.890824 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.890804    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"]
Jul 21 17:06:32.982548 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.982519    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d548995-f3ec-4fda-bcd4-b9418d8a37c9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2qng9\" (UID: \"8d548995-f3ec-4fda-bcd4-b9418d8a37c9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:32.982681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:32.982592    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtgw7\" (UniqueName: \"kubernetes.io/projected/8d548995-f3ec-4fda-bcd4-b9418d8a37c9-kube-api-access-rtgw7\") pod \"openshift-lws-operator-fd8ccff4c-2qng9\" (UID: \"8d548995-f3ec-4fda-bcd4-b9418d8a37c9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:33.085021 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.084192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d548995-f3ec-4fda-bcd4-b9418d8a37c9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2qng9\" (UID: \"8d548995-f3ec-4fda-bcd4-b9418d8a37c9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:33.085021 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.084296    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgw7\" (UniqueName: \"kubernetes.io/projected/8d548995-f3ec-4fda-bcd4-b9418d8a37c9-kube-api-access-rtgw7\") pod \"openshift-lws-operator-fd8ccff4c-2qng9\" (UID: \"8d548995-f3ec-4fda-bcd4-b9418d8a37c9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:33.085021 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.084960    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d548995-f3ec-4fda-bcd4-b9418d8a37c9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2qng9\" (UID: \"8d548995-f3ec-4fda-bcd4-b9418d8a37c9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:33.098882 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.098860    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtgw7\" (UniqueName: \"kubernetes.io/projected/8d548995-f3ec-4fda-bcd4-b9418d8a37c9-kube-api-access-rtgw7\") pod \"openshift-lws-operator-fd8ccff4c-2qng9\" (UID: \"8d548995-f3ec-4fda-bcd4-b9418d8a37c9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:33.185661 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.185639    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"
Jul 21 17:06:33.319704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.319674    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9"]
Jul 21 17:06:33.322177 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:33.322144    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d548995_f3ec_4fda_bcd4_b9418d8a37c9.slice/crio-98c28b04ed2f6b85849f8f5dd83b8f7cccf815f04628dbffa12c67780c437c83 WatchSource:0}: Error finding container 98c28b04ed2f6b85849f8f5dd83b8f7cccf815f04628dbffa12c67780c437c83: Status 404 returned error can't find the container with id 98c28b04ed2f6b85849f8f5dd83b8f7cccf815f04628dbffa12c67780c437c83
Jul 21 17:06:33.648241 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:33.648204    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9" event={"ID":"8d548995-f3ec-4fda-bcd4-b9418d8a37c9","Type":"ContainerStarted","Data":"98c28b04ed2f6b85849f8f5dd83b8f7cccf815f04628dbffa12c67780c437c83"}
Jul 21 17:06:36.664802 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:36.664766    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9" event={"ID":"8d548995-f3ec-4fda-bcd4-b9418d8a37c9","Type":"ContainerStarted","Data":"82bae3946189e1be531eb10cae27965ae9574ef09a933084294a25be5d5c0c98"}
Jul 21 17:06:36.688933 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:36.688871    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qng9" podStartSLOduration=2.229217767 podStartE2EDuration="4.688842695s" totalImagesPullingTime="2.459624928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:36.688531808 +0000 UTC m=+276.653431403" watchObservedRunningTime="2026-07-21 17:06:36.688842695 +0000 UTC m=+276.653742302"
Jul 21 17:06:40.533586 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.519295    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"]
Jul 21 17:06:40.543926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.543892    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"]
Jul 21 17:06:40.544228 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.544205    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.547377 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.547354    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 17:06:40.547504 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.547411    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 17:06:40.548538 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.548251    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p9j9s\""
Jul 21 17:06:40.645017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.644980    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.645017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.645017    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvgkn\" (UniqueName: \"kubernetes.io/projected/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-kube-api-access-zvgkn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.645209 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.645107    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.746089 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.746050    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.746246 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.746103    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvgkn\" (UniqueName: \"kubernetes.io/projected/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-kube-api-access-zvgkn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.746246 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.746169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.746496 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.746474    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.746531 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.746498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.756459 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.756436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvgkn\" (UniqueName: \"kubernetes.io/projected/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-kube-api-access-zvgkn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.855647 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.855549    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:40.986286 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:40.986255    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"]
Jul 21 17:06:40.989407 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:40.989375    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce6791a_ddc7_45b5_ad2f_f4b320f58920.slice/crio-398724e49eeed4e4b5b0491f077f5516a3243ebd78c6cbef3192fed309ca1fac WatchSource:0}: Error finding container 398724e49eeed4e4b5b0491f077f5516a3243ebd78c6cbef3192fed309ca1fac: Status 404 returned error can't find the container with id 398724e49eeed4e4b5b0491f077f5516a3243ebd78c6cbef3192fed309ca1fac
Jul 21 17:06:41.688147 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:41.688112    2561 generic.go:356] "Generic (PLEG): container finished" podID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerID="922229cb39d61195a081e6329130c8f3b4f5930ff8eaee65d92957c8f78dd986" exitCode=0
Jul 21 17:06:41.688534 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:41.688197    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg" event={"ID":"7ce6791a-ddc7-45b5-ad2f-f4b320f58920","Type":"ContainerDied","Data":"922229cb39d61195a081e6329130c8f3b4f5930ff8eaee65d92957c8f78dd986"}
Jul 21 17:06:41.688534 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:41.688244    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg" event={"ID":"7ce6791a-ddc7-45b5-ad2f-f4b320f58920","Type":"ContainerStarted","Data":"398724e49eeed4e4b5b0491f077f5516a3243ebd78c6cbef3192fed309ca1fac"}
Jul 21 17:06:42.694009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:42.693978    2561 generic.go:356] "Generic (PLEG): container finished" podID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerID="8f06e13bf02a06daa5b81027aaec6d8cfb7f01eb91dbbbd949c3633557b1a334" exitCode=0
Jul 21 17:06:42.694392 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:42.694075    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg" event={"ID":"7ce6791a-ddc7-45b5-ad2f-f4b320f58920","Type":"ContainerDied","Data":"8f06e13bf02a06daa5b81027aaec6d8cfb7f01eb91dbbbd949c3633557b1a334"}
Jul 21 17:06:43.700463 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:43.700423    2561 generic.go:356] "Generic (PLEG): container finished" podID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerID="e72fa55cf97967ced9b1be75ac2e14bc06cbd97328e6a13d2914ffbec95110fd" exitCode=0
Jul 21 17:06:43.700864 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:43.700499    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg" event={"ID":"7ce6791a-ddc7-45b5-ad2f-f4b320f58920","Type":"ContainerDied","Data":"e72fa55cf97967ced9b1be75ac2e14bc06cbd97328e6a13d2914ffbec95110fd"}
Jul 21 17:06:44.826987 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.826964    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:44.878041 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.878015    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-bundle\") pod \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") "
Jul 21 17:06:44.878213 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.878052    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvgkn\" (UniqueName: \"kubernetes.io/projected/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-kube-api-access-zvgkn\") pod \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") "
Jul 21 17:06:44.878213 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.878089    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-util\") pod \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\" (UID: \"7ce6791a-ddc7-45b5-ad2f-f4b320f58920\") "
Jul 21 17:06:44.878804 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.878777    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-bundle" (OuterVolumeSpecName: "bundle") pod "7ce6791a-ddc7-45b5-ad2f-f4b320f58920" (UID: "7ce6791a-ddc7-45b5-ad2f-f4b320f58920"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:06:44.880211 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.880188    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-kube-api-access-zvgkn" (OuterVolumeSpecName: "kube-api-access-zvgkn") pod "7ce6791a-ddc7-45b5-ad2f-f4b320f58920" (UID: "7ce6791a-ddc7-45b5-ad2f-f4b320f58920"). InnerVolumeSpecName "kube-api-access-zvgkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:06:44.883247 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.883208    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-util" (OuterVolumeSpecName: "util") pod "7ce6791a-ddc7-45b5-ad2f-f4b320f58920" (UID: "7ce6791a-ddc7-45b5-ad2f-f4b320f58920"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:06:44.979017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.978938    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:44.979017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.978969    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvgkn\" (UniqueName: \"kubernetes.io/projected/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-kube-api-access-zvgkn\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:44.979017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:44.978979    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ce6791a-ddc7-45b5-ad2f-f4b320f58920-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:06:45.709542 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:45.709506    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg" event={"ID":"7ce6791a-ddc7-45b5-ad2f-f4b320f58920","Type":"ContainerDied","Data":"398724e49eeed4e4b5b0491f077f5516a3243ebd78c6cbef3192fed309ca1fac"}
Jul 21 17:06:45.709542 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:45.709542    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpj2pg"
Jul 21 17:06:45.709756 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:45.709574    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="398724e49eeed4e4b5b0491f077f5516a3243ebd78c6cbef3192fed309ca1fac"
Jul 21 17:06:53.288633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288600    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"]
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288877    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="extract"
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288888    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="extract"
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288899    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="util"
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288904    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="util"
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288911    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="pull"
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288917    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="pull"
Jul 21 17:06:53.289069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.288963    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce6791a-ddc7-45b5-ad2f-f4b320f58920" containerName="extract"
Jul 21 17:06:53.291916 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.291899    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.294714 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.294684    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 17:06:53.294838 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.294708    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 17:06:53.294912 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.294900    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 17:06:53.295055 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.295037    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tgbxz\""
Jul 21 17:06:53.295419 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.295400    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 17:06:53.318412 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.318382    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"]
Jul 21 17:06:53.342315 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.342287    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.342417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.342349    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.342417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.342371    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzsbt\" (UniqueName: \"kubernetes.io/projected/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-kube-api-access-rzsbt\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.342417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.342390    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-tmp\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.342417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.342411    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-manifests\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.443161 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.443132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.443308 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.443165    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzsbt\" (UniqueName: \"kubernetes.io/projected/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-kube-api-access-rzsbt\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.443308 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.443183    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-tmp\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.443308 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.443201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-manifests\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.443308 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.443245    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.443742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.443720    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-manifests\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.445764 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.445740    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-tmp\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.445895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.445876    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.445950 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.445877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.469718 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.469688    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzsbt\" (UniqueName: \"kubernetes.io/projected/e9cc9529-59b5-49a2-a7b5-a640d9bfbc11-kube-api-access-rzsbt\") pod \"opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7\" (UID: \"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11\") " pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.602649 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.602544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:53.749057 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:53.749018    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"]
Jul 21 17:06:53.751075 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:06:53.751050    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9cc9529_59b5_49a2_a7b5_a640d9bfbc11.slice/crio-546a7815cd6b14a8070d0824ee2437df10eff2a17aae80a5672a74b403899484 WatchSource:0}: Error finding container 546a7815cd6b14a8070d0824ee2437df10eff2a17aae80a5672a74b403899484: Status 404 returned error can't find the container with id 546a7815cd6b14a8070d0824ee2437df10eff2a17aae80a5672a74b403899484
Jul 21 17:06:54.747707 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:54.747664    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7" event={"ID":"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11","Type":"ContainerStarted","Data":"546a7815cd6b14a8070d0824ee2437df10eff2a17aae80a5672a74b403899484"}
Jul 21 17:06:57.766683 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:57.766630    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7" event={"ID":"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11","Type":"ContainerStarted","Data":"a7f34d7297d2ea27c8b5345533520f51b73be66dd04b52c18c54bc7264a90065"}
Jul 21 17:06:58.772714 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:58.772678    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9cc9529-59b5-49a2-a7b5-a640d9bfbc11" containerID="a7f34d7297d2ea27c8b5345533520f51b73be66dd04b52c18c54bc7264a90065" exitCode=0
Jul 21 17:06:58.773130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:58.772761    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7" event={"ID":"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11","Type":"ContainerDied","Data":"a7f34d7297d2ea27c8b5345533520f51b73be66dd04b52c18c54bc7264a90065"}
Jul 21 17:06:59.779096 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:59.779059    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7" event={"ID":"e9cc9529-59b5-49a2-a7b5-a640d9bfbc11","Type":"ContainerStarted","Data":"ba3e5ca56588fcc89ab07eb8af49ec3409a86353c792871094ac47bb966ce340"}
Jul 21 17:06:59.779536 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:59.779201    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:06:59.807431 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:06:59.807385    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7" podStartSLOduration=2.142434014 podStartE2EDuration="6.807370657s" totalImagesPullingTime="3.582942759s" totalInitContainerRuntime="1.081993884s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:06:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:06:59.805533406 +0000 UTC m=+299.770433009" watchObservedRunningTime="2026-07-21 17:06:59.807370657 +0000 UTC m=+299.772270260"
Jul 21 17:07:00.427855 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:00.427820    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:07:00.428287 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:00.428261    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:07:00.431725 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:00.431706    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 17:07:10.785541 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:10.785508    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7"
Jul 21 17:07:13.122798 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.122758    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-csxsd"]
Jul 21 17:07:13.124800 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.124783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.128007 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.127982    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-t7lwh\""
Jul 21 17:07:13.128145 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.128131    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 21 17:07:13.140497 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.140475    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-csxsd"]
Jul 21 17:07:13.204126 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.204095    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648562a2-3465-4e0a-88f9-0968d6bf7ec2-cert\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.204272 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.204128    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8l5v\" (UniqueName: \"kubernetes.io/projected/648562a2-3465-4e0a-88f9-0968d6bf7ec2-kube-api-access-g8l5v\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.305636 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.305600    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648562a2-3465-4e0a-88f9-0968d6bf7ec2-cert\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.305636 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.305637    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8l5v\" (UniqueName: \"kubernetes.io/projected/648562a2-3465-4e0a-88f9-0968d6bf7ec2-kube-api-access-g8l5v\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.305837 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:07:13.305742    2561 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 17:07:13.305837 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:07:13.305803    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648562a2-3465-4e0a-88f9-0968d6bf7ec2-cert podName:648562a2-3465-4e0a-88f9-0968d6bf7ec2 nodeName:}" failed. No retries permitted until 2026-07-21 17:07:13.805786669 +0000 UTC m=+313.770686251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/648562a2-3465-4e0a-88f9-0968d6bf7ec2-cert") pod "odh-model-controller-7998c67dc4-csxsd" (UID: "648562a2-3465-4e0a-88f9-0968d6bf7ec2") : secret "odh-model-controller-webhook-cert" not found
Jul 21 17:07:13.315520 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.315491    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8l5v\" (UniqueName: \"kubernetes.io/projected/648562a2-3465-4e0a-88f9-0968d6bf7ec2-kube-api-access-g8l5v\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.809865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.809836    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648562a2-3465-4e0a-88f9-0968d6bf7ec2-cert\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:13.812330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:13.812307    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648562a2-3465-4e0a-88f9-0968d6bf7ec2-cert\") pod \"odh-model-controller-7998c67dc4-csxsd\" (UID: \"648562a2-3465-4e0a-88f9-0968d6bf7ec2\") " pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:14.034350 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:14.034320    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:14.167576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:14.167531    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-csxsd"]
Jul 21 17:07:14.169739 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:07:14.169713    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648562a2_3465_4e0a_88f9_0968d6bf7ec2.slice/crio-b3eaa52cee0e8df02ea0fe1e365a93d0c28b823fc662cf92ed5d328651561890 WatchSource:0}: Error finding container b3eaa52cee0e8df02ea0fe1e365a93d0c28b823fc662cf92ed5d328651561890: Status 404 returned error can't find the container with id b3eaa52cee0e8df02ea0fe1e365a93d0c28b823fc662cf92ed5d328651561890
Jul 21 17:07:14.171162 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:14.171144    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:07:14.849446 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:14.849403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd" event={"ID":"648562a2-3465-4e0a-88f9-0968d6bf7ec2","Type":"ContainerStarted","Data":"b3eaa52cee0e8df02ea0fe1e365a93d0c28b823fc662cf92ed5d328651561890"}
Jul 21 17:07:16.269929 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.269896    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"]
Jul 21 17:07:16.273258 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.273239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.278774 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.278750    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 17:07:16.278917 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.278896    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 17:07:16.279989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.279800    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 17:07:16.279989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.279875    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 17:07:16.279989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.279943    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9g22j\""
Jul 21 17:07:16.289190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.289169    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"]
Jul 21 17:07:16.434603 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.434545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jd5\" (UniqueName: \"kubernetes.io/projected/59a3d206-484b-453e-aafd-d0d2936c2989-kube-api-access-k4jd5\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.434781 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.434611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a3d206-484b-453e-aafd-d0d2936c2989-tmp\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.434781 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.434639    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a3d206-484b-453e-aafd-d0d2936c2989-tls-certs\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.535763 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.535684    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jd5\" (UniqueName: \"kubernetes.io/projected/59a3d206-484b-453e-aafd-d0d2936c2989-kube-api-access-k4jd5\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.535763 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.535728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a3d206-484b-453e-aafd-d0d2936c2989-tmp\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.535763 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.535755    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a3d206-484b-453e-aafd-d0d2936c2989-tls-certs\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.538281 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.538255    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a3d206-484b-453e-aafd-d0d2936c2989-tmp\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.538408 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.538324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59a3d206-484b-453e-aafd-d0d2936c2989-tls-certs\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.547937 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.547913    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jd5\" (UniqueName: \"kubernetes.io/projected/59a3d206-484b-453e-aafd-d0d2936c2989-kube-api-access-k4jd5\") pod \"kube-auth-proxy-66868cbd5-dl2ln\" (UID: \"59a3d206-484b-453e-aafd-d0d2936c2989\") " pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:16.584334 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:16.584311    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"
Jul 21 17:07:17.188669 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.188643    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln"]
Jul 21 17:07:17.190669 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:07:17.190638    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a3d206_484b_453e_aafd_d0d2936c2989.slice/crio-ea00f9f235910200f600813a1d22d4a1baf9a54ab66788d6497252624e903d81 WatchSource:0}: Error finding container ea00f9f235910200f600813a1d22d4a1baf9a54ab66788d6497252624e903d81: Status 404 returned error can't find the container with id ea00f9f235910200f600813a1d22d4a1baf9a54ab66788d6497252624e903d81
Jul 21 17:07:17.641924 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.641891    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"]
Jul 21 17:07:17.644238 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.644222    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.651211 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.651180    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 17:07:17.651329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.651233    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 17:07:17.651329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.651289    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 17:07:17.651329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.651314    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 17:07:17.651329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.651233    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-72t6n\""
Jul 21 17:07:17.665347 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.665316    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"]
Jul 21 17:07:17.746303 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smdbt\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-kube-api-access-smdbt\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.746500 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746317    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.746500 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746387    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.746500 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746435    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.746500 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746480    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6395c774-e6aa-4f51-8b17-77734bbcb11d-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.746749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746513    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.746749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.746669    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smdbt\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-kube-api-access-smdbt\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847577    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847609    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847634    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6395c774-e6aa-4f51-8b17-77734bbcb11d-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.847704 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.847702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.848410 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.848370    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.850787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.850759    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.851265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.850988    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6395c774-e6aa-4f51-8b17-77734bbcb11d-local-certs\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.851265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.851161    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-cacerts\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.851265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.851223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.857546 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.857523    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smdbt\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-kube-api-access-smdbt\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.857766 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.857746    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-token\") pod \"istiod-openshift-gateway-7574459656-m7twt\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:17.867578 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.867526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd" event={"ID":"648562a2-3465-4e0a-88f9-0968d6bf7ec2","Type":"ContainerStarted","Data":"ca42f3ff5f6a5fab193f1726ffd98d8d7a48298142d1c5c4a7277d79985a56d0"}
Jul 21 17:07:17.867681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.867616    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:07:17.869180 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.869146    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln" event={"ID":"59a3d206-484b-453e-aafd-d0d2936c2989","Type":"ContainerStarted","Data":"ea00f9f235910200f600813a1d22d4a1baf9a54ab66788d6497252624e903d81"}
Jul 21 17:07:17.888373 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.888324    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd" podStartSLOduration=1.988971399 podStartE2EDuration="4.888306436s" totalImagesPullingTime="2.899335037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:17.887715499 +0000 UTC m=+317.852615104" watchObservedRunningTime="2026-07-21 17:07:17.888306436 +0000 UTC m=+317.853206041"
Jul 21 17:07:17.954422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:17.954339    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:18.108966 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:18.108907    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"]
Jul 21 17:07:18.111666 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:07:18.111638    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6395c774_e6aa_4f51_8b17_77734bbcb11d.slice/crio-711f15a2b54669b436ea3ea2484bf8822f12a04377f56c78ca5a2e068c7e254d WatchSource:0}: Error finding container 711f15a2b54669b436ea3ea2484bf8822f12a04377f56c78ca5a2e068c7e254d: Status 404 returned error can't find the container with id 711f15a2b54669b436ea3ea2484bf8822f12a04377f56c78ca5a2e068c7e254d
Jul 21 17:07:18.873919 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:18.873872    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt" event={"ID":"6395c774-e6aa-4f51-8b17-77734bbcb11d","Type":"ContainerStarted","Data":"711f15a2b54669b436ea3ea2484bf8822f12a04377f56c78ca5a2e068c7e254d"}
Jul 21 17:07:21.116137 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.116102    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 21 17:07:21.116382 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.116169    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 21 17:07:21.670285 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.670248    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"]
Jul 21 17:07:21.673568 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.673533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.719511 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.719481    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"]
Jul 21 17:07:21.783404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.783404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783403    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.783633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783438    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/1c3fec08-4d01-4c93-be47-906d0fe10b1b-kube-api-access-gnb2m\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.783633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783516    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.783633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783608    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3fec08-4d01-4c93-be47-906d0fe10b1b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.783633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783627    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.783783 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.783665    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885118 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.884869    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885318 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.885174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3fec08-4d01-4c93-be47-906d0fe10b1b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885318 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.885214    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885318 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.885278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885532 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.885354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885532 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.885381    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.885532 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.885426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/1c3fec08-4d01-4c93-be47-906d0fe10b1b-kube-api-access-gnb2m\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.886345 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.886318    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.888495 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.888470    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.888629 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.888548    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3fec08-4d01-4c93-be47-906d0fe10b1b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.888728 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.888705    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.889099 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.889077    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.894428 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.894353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln" event={"ID":"59a3d206-484b-453e-aafd-d0d2936c2989","Type":"ContainerStarted","Data":"5b7f9061e9dcceb988a42687469f7a2b64d56bdff4235951214deac43c738301"}
Jul 21 17:07:21.896184 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.896159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt" event={"ID":"6395c774-e6aa-4f51-8b17-77734bbcb11d","Type":"ContainerStarted","Data":"405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551"}
Jul 21 17:07:21.896316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.896300    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:21.904000 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.903969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3fec08-4d01-4c93-be47-906d0fe10b1b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.906349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.906330    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnb2m\" (UniqueName: \"kubernetes.io/projected/1c3fec08-4d01-4c93-be47-906d0fe10b1b-kube-api-access-gnb2m\") pod \"istiod-openshift-gateway-94bb8fbfd-gdp2p\" (UID: \"1c3fec08-4d01-4c93-be47-906d0fe10b1b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:21.924510 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.924438    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66868cbd5-dl2ln" podStartSLOduration=2.004233055 podStartE2EDuration="5.924424782s" totalImagesPullingTime="3.920191727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:21.92241475 +0000 UTC m=+321.887314356" watchObservedRunningTime="2026-07-21 17:07:21.924424782 +0000 UTC m=+321.889324386"
Jul 21 17:07:21.953006 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.952949    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt" podStartSLOduration=1.9507118719999998 podStartE2EDuration="4.952930425s" totalImagesPullingTime="3.002218553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:21.95101049 +0000 UTC m=+321.915910094" watchObservedRunningTime="2026-07-21 17:07:21.952930425 +0000 UTC m=+321.917830030"
Jul 21 17:07:21.983384 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:21.983360    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:22.124435 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.124358    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"]
Jul 21 17:07:22.136587 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.135940    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 21 17:07:22.136587 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.136027    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 21 17:07:22.903107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.903073    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p" event={"ID":"1c3fec08-4d01-4c93-be47-906d0fe10b1b","Type":"ContainerStarted","Data":"528fc44413810fc2fc036358f8f032460be97526b9a1740cb2682de4dfeab1cd"}
Jul 21 17:07:22.903107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.903124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p" event={"ID":"1c3fec08-4d01-4c93-be47-906d0fe10b1b","Type":"ContainerStarted","Data":"6b1f9bf6c3a0b1f39d19d58d76c73f65e0e3f895acfc0f8cae35eecd638db048"}
Jul 21 17:07:22.903354 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.903166    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:22.905227 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:22.905207    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:23.074999 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:23.074944    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p" podStartSLOduration=2.074925776 podStartE2EDuration="2.074925776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:07:22.987738592 +0000 UTC m=+322.952638195" watchObservedRunningTime="2026-07-21 17:07:23.074925776 +0000 UTC m=+323.039825379"
Jul 21 17:07:23.908997 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:23.908968    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gdp2p"
Jul 21 17:07:23.977208 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:23.977178    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"]
Jul 21 17:07:24.911603 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:24.911551    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt" podUID="6395c774-e6aa-4f51-8b17-77734bbcb11d" containerName="discovery" containerID="cri-o://405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551" gracePeriod=30
Jul 21 17:07:25.163858 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.163806    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:25.319343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319312    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smdbt\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-kube-api-access-smdbt\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319349    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-kubeconfig\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319379    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-ca-configmap\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319400    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6395c774-e6aa-4f51-8b17-77734bbcb11d-local-certs\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319461    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-cacerts\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319491    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-dns-cert\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319515    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-token\") pod \"6395c774-e6aa-4f51-8b17-77734bbcb11d\" (UID: \"6395c774-e6aa-4f51-8b17-77734bbcb11d\") "
Jul 21 17:07:25.319879 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.319796    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:07:25.321838 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.321803    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:07:25.321954 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.321929    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-token" (OuterVolumeSpecName: "istio-token") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:07:25.322402 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.322350    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-kube-api-access-smdbt" (OuterVolumeSpecName: "kube-api-access-smdbt") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "kube-api-access-smdbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:07:25.322402 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.322368    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6395c774-e6aa-4f51-8b17-77734bbcb11d-local-certs" (OuterVolumeSpecName: "local-certs") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:07:25.322548 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.322425    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-cacerts" (OuterVolumeSpecName: "cacerts") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:07:25.322548 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.322437    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6395c774-e6aa-4f51-8b17-77734bbcb11d" (UID: "6395c774-e6aa-4f51-8b17-77734bbcb11d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420409    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smdbt\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-kube-api-access-smdbt\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420440    2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-kubeconfig\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420449    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-ca-configmap\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420457    2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6395c774-e6aa-4f51-8b17-77734bbcb11d-local-certs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420466    2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-cacerts\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420474    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-csr-dns-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.420494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.420483    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6395c774-e6aa-4f51-8b17-77734bbcb11d-istio-token\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:07:25.916857 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.916821    2561 generic.go:356] "Generic (PLEG): container finished" podID="6395c774-e6aa-4f51-8b17-77734bbcb11d" containerID="405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551" exitCode=0
Jul 21 17:07:25.917250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.916861    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"
Jul 21 17:07:25.917250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.916903    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt" event={"ID":"6395c774-e6aa-4f51-8b17-77734bbcb11d","Type":"ContainerDied","Data":"405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551"}
Jul 21 17:07:25.917250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.916959    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-m7twt" event={"ID":"6395c774-e6aa-4f51-8b17-77734bbcb11d","Type":"ContainerDied","Data":"711f15a2b54669b436ea3ea2484bf8822f12a04377f56c78ca5a2e068c7e254d"}
Jul 21 17:07:25.917250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.916982    2561 scope.go:117] "RemoveContainer" containerID="405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551"
Jul 21 17:07:25.925775 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.925759    2561 scope.go:117] "RemoveContainer" containerID="405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551"
Jul 21 17:07:25.926037 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:07:25.926019    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551\": container with ID starting with 405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551 not found: ID does not exist" containerID="405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551"
Jul 21 17:07:25.926090 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.926044    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551"} err="failed to get container status \"405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551\": rpc error: code = NotFound desc = could not find container \"405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551\": container with ID starting with 405b0dafdb161588592552fa601da315e8ebb6bac3df0c8cfefa647193de9551 not found: ID does not exist"
Jul 21 17:07:25.942666 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.942635    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"]
Jul 21 17:07:25.945652 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:25.945632    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-m7twt"]
Jul 21 17:07:26.513440 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:26.513403    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6395c774-e6aa-4f51-8b17-77734bbcb11d" path="/var/lib/kubelet/pods/6395c774-e6aa-4f51-8b17-77734bbcb11d/volumes"
Jul 21 17:07:28.876326 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:07:28.876297    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-csxsd"
Jul 21 17:08:14.107122 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.107086    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tm2d6"]
Jul 21 17:08:14.107509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.107404    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6395c774-e6aa-4f51-8b17-77734bbcb11d" containerName="discovery"
Jul 21 17:08:14.107509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.107414    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6395c774-e6aa-4f51-8b17-77734bbcb11d" containerName="discovery"
Jul 21 17:08:14.107509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.107463    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6395c774-e6aa-4f51-8b17-77734bbcb11d" containerName="discovery"
Jul 21 17:08:14.111176 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.111160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:14.114784 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.114760    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-wdsgb\""
Jul 21 17:08:14.114927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.114854    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 17:08:14.115570 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.115543    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 17:08:14.136227 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.136196    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tm2d6"]
Jul 21 17:08:14.215983 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.215954    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4qrt\" (UniqueName: \"kubernetes.io/projected/200e0cdc-b91f-4b01-8511-55818ec7e085-kube-api-access-b4qrt\") pod \"kuadrant-operator-catalog-tm2d6\" (UID: \"200e0cdc-b91f-4b01-8511-55818ec7e085\") " pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:14.317221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.317192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4qrt\" (UniqueName: \"kubernetes.io/projected/200e0cdc-b91f-4b01-8511-55818ec7e085-kube-api-access-b4qrt\") pod \"kuadrant-operator-catalog-tm2d6\" (UID: \"200e0cdc-b91f-4b01-8511-55818ec7e085\") " pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:14.327110 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.327081    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4qrt\" (UniqueName: \"kubernetes.io/projected/200e0cdc-b91f-4b01-8511-55818ec7e085-kube-api-access-b4qrt\") pod \"kuadrant-operator-catalog-tm2d6\" (UID: \"200e0cdc-b91f-4b01-8511-55818ec7e085\") " pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:14.419926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.419901    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:14.420749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.420728    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tm2d6"]
Jul 21 17:08:14.549895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.549870    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tm2d6"]
Jul 21 17:08:14.552256 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:14.552230    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200e0cdc_b91f_4b01_8511_55818ec7e085.slice/crio-d0ac604b04f40fb9fd42a94709e8c0227291bdd3ca82a642707e4bd5991ab9a9 WatchSource:0}: Error finding container d0ac604b04f40fb9fd42a94709e8c0227291bdd3ca82a642707e4bd5991ab9a9: Status 404 returned error can't find the container with id d0ac604b04f40fb9fd42a94709e8c0227291bdd3ca82a642707e4bd5991ab9a9
Jul 21 17:08:14.632836 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.632809    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vzkmm"]
Jul 21 17:08:14.635453 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.635438    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:14.645639 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.645615    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vzkmm"]
Jul 21 17:08:14.721213 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.721151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xs5x\" (UniqueName: \"kubernetes.io/projected/1a2a4b3d-6789-4781-b127-d8aaebfac6a1-kube-api-access-7xs5x\") pod \"kuadrant-operator-catalog-vzkmm\" (UID: \"1a2a4b3d-6789-4781-b127-d8aaebfac6a1\") " pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:14.822218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.822187    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xs5x\" (UniqueName: \"kubernetes.io/projected/1a2a4b3d-6789-4781-b127-d8aaebfac6a1-kube-api-access-7xs5x\") pod \"kuadrant-operator-catalog-vzkmm\" (UID: \"1a2a4b3d-6789-4781-b127-d8aaebfac6a1\") " pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:14.833130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.833104    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xs5x\" (UniqueName: \"kubernetes.io/projected/1a2a4b3d-6789-4781-b127-d8aaebfac6a1-kube-api-access-7xs5x\") pod \"kuadrant-operator-catalog-vzkmm\" (UID: \"1a2a4b3d-6789-4781-b127-d8aaebfac6a1\") " pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:14.944963 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:14.944941    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:15.075139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:15.075106    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vzkmm"]
Jul 21 17:08:15.076961 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:15.076936    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2a4b3d_6789_4781_b127_d8aaebfac6a1.slice/crio-004fe05d7edd6b2260946c6dcf263dbc8657a8d3566dca8447a9bdcc32286cb4 WatchSource:0}: Error finding container 004fe05d7edd6b2260946c6dcf263dbc8657a8d3566dca8447a9bdcc32286cb4: Status 404 returned error can't find the container with id 004fe05d7edd6b2260946c6dcf263dbc8657a8d3566dca8447a9bdcc32286cb4
Jul 21 17:08:15.144452 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:15.144425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm" event={"ID":"1a2a4b3d-6789-4781-b127-d8aaebfac6a1","Type":"ContainerStarted","Data":"004fe05d7edd6b2260946c6dcf263dbc8657a8d3566dca8447a9bdcc32286cb4"}
Jul 21 17:08:15.145523 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:15.145491    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6" event={"ID":"200e0cdc-b91f-4b01-8511-55818ec7e085","Type":"ContainerStarted","Data":"d0ac604b04f40fb9fd42a94709e8c0227291bdd3ca82a642707e4bd5991ab9a9"}
Jul 21 17:08:17.157642 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.157604    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6" event={"ID":"200e0cdc-b91f-4b01-8511-55818ec7e085","Type":"ContainerStarted","Data":"d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1"}
Jul 21 17:08:17.158125 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.157713    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6" podUID="200e0cdc-b91f-4b01-8511-55818ec7e085" containerName="registry-server" containerID="cri-o://d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1" gracePeriod=2
Jul 21 17:08:17.159256 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.159221    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm" event={"ID":"1a2a4b3d-6789-4781-b127-d8aaebfac6a1","Type":"ContainerStarted","Data":"7edc05e50dec1df866b99faefe857e5939b5c7cb7b2d4e36e30c10b6f35c2061"}
Jul 21 17:08:17.183150 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.183107    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6" podStartSLOduration=0.917126374 podStartE2EDuration="3.183093934s" totalImagesPullingTime="2.26596756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:17.180637051 +0000 UTC m=+377.145536655" watchObservedRunningTime="2026-07-21 17:08:17.183093934 +0000 UTC m=+377.147993537"
Jul 21 17:08:17.208474 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.208427    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm" podStartSLOduration=1.489967354 podStartE2EDuration="3.208411675s" totalImagesPullingTime="1.718444321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:17.20766955 +0000 UTC m=+377.172569166" watchObservedRunningTime="2026-07-21 17:08:17.208411675 +0000 UTC m=+377.173311279"
Jul 21 17:08:17.392204 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.392178    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:17.550222 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.550130    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4qrt\" (UniqueName: \"kubernetes.io/projected/200e0cdc-b91f-4b01-8511-55818ec7e085-kube-api-access-b4qrt\") pod \"200e0cdc-b91f-4b01-8511-55818ec7e085\" (UID: \"200e0cdc-b91f-4b01-8511-55818ec7e085\") "
Jul 21 17:08:17.552449 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.552422    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200e0cdc-b91f-4b01-8511-55818ec7e085-kube-api-access-b4qrt" (OuterVolumeSpecName: "kube-api-access-b4qrt") pod "200e0cdc-b91f-4b01-8511-55818ec7e085" (UID: "200e0cdc-b91f-4b01-8511-55818ec7e085"). InnerVolumeSpecName "kube-api-access-b4qrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:08:17.651629 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:17.651581    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4qrt\" (UniqueName: \"kubernetes.io/projected/200e0cdc-b91f-4b01-8511-55818ec7e085-kube-api-access-b4qrt\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:18.165219 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.165184    2561 generic.go:356] "Generic (PLEG): container finished" podID="200e0cdc-b91f-4b01-8511-55818ec7e085" containerID="d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1" exitCode=0
Jul 21 17:08:18.165675 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.165230    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6"
Jul 21 17:08:18.165675 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.165271    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6" event={"ID":"200e0cdc-b91f-4b01-8511-55818ec7e085","Type":"ContainerDied","Data":"d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1"}
Jul 21 17:08:18.165675 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.165318    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tm2d6" event={"ID":"200e0cdc-b91f-4b01-8511-55818ec7e085","Type":"ContainerDied","Data":"d0ac604b04f40fb9fd42a94709e8c0227291bdd3ca82a642707e4bd5991ab9a9"}
Jul 21 17:08:18.165675 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.165339    2561 scope.go:117] "RemoveContainer" containerID="d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1"
Jul 21 17:08:18.174938 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.174914    2561 scope.go:117] "RemoveContainer" containerID="d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1"
Jul 21 17:08:18.175195 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:08:18.175178    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1\": container with ID starting with d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1 not found: ID does not exist" containerID="d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1"
Jul 21 17:08:18.175284 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.175204    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1"} err="failed to get container status \"d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1\": rpc error: code = NotFound desc = could not find container \"d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1\": container with ID starting with d460f58a95cbb3be787dbad741037b102d9710632d37e095fad16f18ee64d5d1 not found: ID does not exist"
Jul 21 17:08:18.189993 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.189970    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tm2d6"]
Jul 21 17:08:18.194763 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.194743    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tm2d6"]
Jul 21 17:08:18.513312 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:18.513236    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="200e0cdc-b91f-4b01-8511-55818ec7e085" path="/var/lib/kubelet/pods/200e0cdc-b91f-4b01-8511-55818ec7e085/volumes"
Jul 21 17:08:24.945182 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:24.945146    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:24.945536 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:24.945220    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:24.968006 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:24.967978    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:25.219837 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:25.219751    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vzkmm"
Jul 21 17:08:29.621800 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.621767    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"]
Jul 21 17:08:29.622274 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.622255    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200e0cdc-b91f-4b01-8511-55818ec7e085" containerName="registry-server"
Jul 21 17:08:29.622274 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.622274    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="200e0cdc-b91f-4b01-8511-55818ec7e085" containerName="registry-server"
Jul 21 17:08:29.622386 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.622372    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="200e0cdc-b91f-4b01-8511-55818ec7e085" containerName="registry-server"
Jul 21 17:08:29.625416 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.625395    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.627740 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.627724    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kmzmg\""
Jul 21 17:08:29.633886 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.633867    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"]
Jul 21 17:08:29.751072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.751040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.751272 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.751080    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.751272 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.751161    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9z42\" (UniqueName: \"kubernetes.io/projected/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-kube-api-access-c9z42\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.852319 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.852288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9z42\" (UniqueName: \"kubernetes.io/projected/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-kube-api-access-c9z42\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.852410 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.852351    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.852410 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.852373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.852701 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.852686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.852740 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.852727    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.861684 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.861664    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9z42\" (UniqueName: \"kubernetes.io/projected/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-kube-api-access-c9z42\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:29.935326 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:29.935300    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:30.067526 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.067500    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"]
Jul 21 17:08:30.070067 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:30.070044    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e56816_ddf7_4baf_8dae_961e2b0ec61f.slice/crio-bf479fb9fd631108086eb9db1a1a55c18cd31faddd13dc709e45f19ac2891af4 WatchSource:0}: Error finding container bf479fb9fd631108086eb9db1a1a55c18cd31faddd13dc709e45f19ac2891af4: Status 404 returned error can't find the container with id bf479fb9fd631108086eb9db1a1a55c18cd31faddd13dc709e45f19ac2891af4
Jul 21 17:08:30.221333 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.221268    2561 generic.go:356] "Generic (PLEG): container finished" podID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerID="b6a4ae7f9333ecf115ac274683c06646808ce0bf899cb85071bebd4293f2059c" exitCode=0
Jul 21 17:08:30.221441 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.221344    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l" event={"ID":"f7e56816-ddf7-4baf-8dae-961e2b0ec61f","Type":"ContainerDied","Data":"b6a4ae7f9333ecf115ac274683c06646808ce0bf899cb85071bebd4293f2059c"}
Jul 21 17:08:30.221441 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.221394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l" event={"ID":"f7e56816-ddf7-4baf-8dae-961e2b0ec61f","Type":"ContainerStarted","Data":"bf479fb9fd631108086eb9db1a1a55c18cd31faddd13dc709e45f19ac2891af4"}
Jul 21 17:08:30.419916 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.419878    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"]
Jul 21 17:08:30.423201 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.423179    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.431315 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.431295    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"]
Jul 21 17:08:30.558344 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.558279    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.558466 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.558350    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.558466 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.558388    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghw2\" (UniqueName: \"kubernetes.io/projected/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-kube-api-access-mghw2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.659287 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.659257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.659727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.659367    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.659727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.659400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghw2\" (UniqueName: \"kubernetes.io/projected/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-kube-api-access-mghw2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.659727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.659684    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.659852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.659750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.670041 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.670015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghw2\" (UniqueName: \"kubernetes.io/projected/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-kube-api-access-mghw2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.732546 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.732520    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:30.826408 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.826371    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"]
Jul 21 17:08:30.830982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.830956    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:30.841663 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.841638    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"]
Jul 21 17:08:30.864610 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.864589    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"]
Jul 21 17:08:30.866090 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:30.866056    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7a5b31_bd95_4d61_b9ee_c41973b4bfc8.slice/crio-5432bc32a6d0fb6b84dc8272e32da7e39bd35abbed9737f5aa054e103d5ea11f WatchSource:0}: Error finding container 5432bc32a6d0fb6b84dc8272e32da7e39bd35abbed9737f5aa054e103d5ea11f: Status 404 returned error can't find the container with id 5432bc32a6d0fb6b84dc8272e32da7e39bd35abbed9737f5aa054e103d5ea11f
Jul 21 17:08:30.961133 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.961099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkplc\" (UniqueName: \"kubernetes.io/projected/8e055721-2549-419e-974e-bc59e120c7db-kube-api-access-kkplc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:30.961265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.961148    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:30.961265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:30.961227    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.061893 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.061820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkplc\" (UniqueName: \"kubernetes.io/projected/8e055721-2549-419e-974e-bc59e120c7db-kube-api-access-kkplc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.061893 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.061869    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.062057 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.061921    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.062239 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.062219    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.062292 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.062275    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.071475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.071440    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkplc\" (UniqueName: \"kubernetes.io/projected/8e055721-2549-419e-974e-bc59e120c7db-kube-api-access-kkplc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.142537 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.142518    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:31.228167 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.228139    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"]
Jul 21 17:08:31.231661 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.231636    2561 generic.go:356] "Generic (PLEG): container finished" podID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerID="b534b0a060211139b474e17a71f7c882d718fe83f4560820f123c4befa984946" exitCode=0
Jul 21 17:08:31.232390 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.232330    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l" event={"ID":"f7e56816-ddf7-4baf-8dae-961e2b0ec61f","Type":"ContainerStarted","Data":"1d9b4d82e4231e953aeb04c1287be7575e4250541f07dc45c9af85499d9efbc8"}
Jul 21 17:08:31.233131 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.232531    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.233322 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.232745    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl" event={"ID":"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8","Type":"ContainerDied","Data":"b534b0a060211139b474e17a71f7c882d718fe83f4560820f123c4befa984946"}
Jul 21 17:08:31.233385 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.233337    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl" event={"ID":"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8","Type":"ContainerStarted","Data":"5432bc32a6d0fb6b84dc8272e32da7e39bd35abbed9737f5aa054e103d5ea11f"}
Jul 21 17:08:31.246744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.246714    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"]
Jul 21 17:08:31.295687 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.295472    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"]
Jul 21 17:08:31.297630 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:31.297607    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e055721_2549_419e_974e_bc59e120c7db.slice/crio-302dec33d620d997dc15656bbafb004e38835f4e4ce77d7a068c3b5fdec7204c WatchSource:0}: Error finding container 302dec33d620d997dc15656bbafb004e38835f4e4ce77d7a068c3b5fdec7204c: Status 404 returned error can't find the container with id 302dec33d620d997dc15656bbafb004e38835f4e4ce77d7a068c3b5fdec7204c
Jul 21 17:08:31.364166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.364128    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.364285 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.364229    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lms26\" (UniqueName: \"kubernetes.io/projected/f59109e5-f725-4588-b6eb-fb632500afb5-kube-api-access-lms26\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.364346 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.364329    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.465745 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.465714    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lms26\" (UniqueName: \"kubernetes.io/projected/f59109e5-f725-4588-b6eb-fb632500afb5-kube-api-access-lms26\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.465889 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.465766    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.465889 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.465818    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.466167 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.466150    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.466213 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.466164    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.475653 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.475633    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lms26\" (UniqueName: \"kubernetes.io/projected/f59109e5-f725-4588-b6eb-fb632500afb5-kube-api-access-lms26\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.585373 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.585281    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:31.726602 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:31.726543    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"]
Jul 21 17:08:31.729737 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:31.729706    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59109e5_f725_4588_b6eb_fb632500afb5.slice/crio-1a8d1fe28a6c06b62a5395f03cbd8f3ff0b63560b04db84760669f0377b63e00 WatchSource:0}: Error finding container 1a8d1fe28a6c06b62a5395f03cbd8f3ff0b63560b04db84760669f0377b63e00: Status 404 returned error can't find the container with id 1a8d1fe28a6c06b62a5395f03cbd8f3ff0b63560b04db84760669f0377b63e00
Jul 21 17:08:32.238542 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.238457    2561 generic.go:356] "Generic (PLEG): container finished" podID="f59109e5-f725-4588-b6eb-fb632500afb5" containerID="40e8567f3eab5ab1be98b60850b28969d5475bac2739f66c1673c49144e1ebdf" exitCode=0
Jul 21 17:08:32.238785 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.238549    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b" event={"ID":"f59109e5-f725-4588-b6eb-fb632500afb5","Type":"ContainerDied","Data":"40e8567f3eab5ab1be98b60850b28969d5475bac2739f66c1673c49144e1ebdf"}
Jul 21 17:08:32.238785 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.238625    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b" event={"ID":"f59109e5-f725-4588-b6eb-fb632500afb5","Type":"ContainerStarted","Data":"1a8d1fe28a6c06b62a5395f03cbd8f3ff0b63560b04db84760669f0377b63e00"}
Jul 21 17:08:32.240639 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.240617    2561 generic.go:356] "Generic (PLEG): container finished" podID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerID="015610184cc7995fbdef111edcdf5d06f3ee06ee13056e3e828cf2aacde739ad" exitCode=0
Jul 21 17:08:32.240724 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.240640    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl" event={"ID":"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8","Type":"ContainerDied","Data":"015610184cc7995fbdef111edcdf5d06f3ee06ee13056e3e828cf2aacde739ad"}
Jul 21 17:08:32.242263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.242245    2561 generic.go:356] "Generic (PLEG): container finished" podID="8e055721-2549-419e-974e-bc59e120c7db" containerID="4621e5d49d870095dea14c87aeb3d5644d92075b67468356feb357a51c09f796" exitCode=0
Jul 21 17:08:32.242352 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.242331    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb" event={"ID":"8e055721-2549-419e-974e-bc59e120c7db","Type":"ContainerDied","Data":"4621e5d49d870095dea14c87aeb3d5644d92075b67468356feb357a51c09f796"}
Jul 21 17:08:32.242415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.242378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb" event={"ID":"8e055721-2549-419e-974e-bc59e120c7db","Type":"ContainerStarted","Data":"302dec33d620d997dc15656bbafb004e38835f4e4ce77d7a068c3b5fdec7204c"}
Jul 21 17:08:32.244843 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.244826    2561 generic.go:356] "Generic (PLEG): container finished" podID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerID="1d9b4d82e4231e953aeb04c1287be7575e4250541f07dc45c9af85499d9efbc8" exitCode=0
Jul 21 17:08:32.244843 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.244840    2561 generic.go:356] "Generic (PLEG): container finished" podID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerID="02a39c8f1c96270fcfe15a0b7098bc3588c4dca75d9f770ba9dc373cf46d7855" exitCode=0
Jul 21 17:08:32.244970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.244874    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l" event={"ID":"f7e56816-ddf7-4baf-8dae-961e2b0ec61f","Type":"ContainerDied","Data":"1d9b4d82e4231e953aeb04c1287be7575e4250541f07dc45c9af85499d9efbc8"}
Jul 21 17:08:32.244970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:32.244903    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l" event={"ID":"f7e56816-ddf7-4baf-8dae-961e2b0ec61f","Type":"ContainerDied","Data":"02a39c8f1c96270fcfe15a0b7098bc3588c4dca75d9f770ba9dc373cf46d7855"}
Jul 21 17:08:33.252353 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.252320    2561 generic.go:356] "Generic (PLEG): container finished" podID="f59109e5-f725-4588-b6eb-fb632500afb5" containerID="289766f0a48658051a0182a36cd3fcc76a44622bdbd747da3578e04a5cd1c5cf" exitCode=0
Jul 21 17:08:33.252787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.252402    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b" event={"ID":"f59109e5-f725-4588-b6eb-fb632500afb5","Type":"ContainerDied","Data":"289766f0a48658051a0182a36cd3fcc76a44622bdbd747da3578e04a5cd1c5cf"}
Jul 21 17:08:33.254506 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.254486    2561 generic.go:356] "Generic (PLEG): container finished" podID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerID="75c59747ca62bb58c129199274fed61cb78aef856b669e162884b230d1dcb82b" exitCode=0
Jul 21 17:08:33.254631 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.254576    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl" event={"ID":"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8","Type":"ContainerDied","Data":"75c59747ca62bb58c129199274fed61cb78aef856b669e162884b230d1dcb82b"}
Jul 21 17:08:33.394378 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.394295    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:33.585117 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.585083    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-bundle\") pod \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") "
Jul 21 17:08:33.585117 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.585124    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-util\") pod \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") "
Jul 21 17:08:33.585318 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.585226    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9z42\" (UniqueName: \"kubernetes.io/projected/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-kube-api-access-c9z42\") pod \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\" (UID: \"f7e56816-ddf7-4baf-8dae-961e2b0ec61f\") "
Jul 21 17:08:33.585777 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.585735    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-bundle" (OuterVolumeSpecName: "bundle") pod "f7e56816-ddf7-4baf-8dae-961e2b0ec61f" (UID: "f7e56816-ddf7-4baf-8dae-961e2b0ec61f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:33.587399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.587377    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-kube-api-access-c9z42" (OuterVolumeSpecName: "kube-api-access-c9z42") pod "f7e56816-ddf7-4baf-8dae-961e2b0ec61f" (UID: "f7e56816-ddf7-4baf-8dae-961e2b0ec61f"). InnerVolumeSpecName "kube-api-access-c9z42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:08:33.590150 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.590116    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-util" (OuterVolumeSpecName: "util") pod "f7e56816-ddf7-4baf-8dae-961e2b0ec61f" (UID: "f7e56816-ddf7-4baf-8dae-961e2b0ec61f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:33.686514 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.686473    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:33.686514 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.686514    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:33.686809 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:33.686532    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9z42\" (UniqueName: \"kubernetes.io/projected/f7e56816-ddf7-4baf-8dae-961e2b0ec61f-kube-api-access-c9z42\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:34.265354 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.265301    2561 generic.go:356] "Generic (PLEG): container finished" podID="f59109e5-f725-4588-b6eb-fb632500afb5" containerID="dcb90a150815d9ae5985ad8cbcb86a1f9c04525f4e20236632575bc26c887cb7" exitCode=0
Jul 21 17:08:34.265845 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.265371    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b" event={"ID":"f59109e5-f725-4588-b6eb-fb632500afb5","Type":"ContainerDied","Data":"dcb90a150815d9ae5985ad8cbcb86a1f9c04525f4e20236632575bc26c887cb7"}
Jul 21 17:08:34.268059 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.268035    2561 generic.go:356] "Generic (PLEG): container finished" podID="8e055721-2549-419e-974e-bc59e120c7db" containerID="b08acb5605aabeaf52af6475711a0adfe4340a8eb8a750e76e875b5ccf12f7a7" exitCode=0
Jul 21 17:08:34.268171 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.268114    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb" event={"ID":"8e055721-2549-419e-974e-bc59e120c7db","Type":"ContainerDied","Data":"b08acb5605aabeaf52af6475711a0adfe4340a8eb8a750e76e875b5ccf12f7a7"}
Jul 21 17:08:34.270161 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.270137    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l"
Jul 21 17:08:34.270236 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.270172    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l" event={"ID":"f7e56816-ddf7-4baf-8dae-961e2b0ec61f","Type":"ContainerDied","Data":"bf479fb9fd631108086eb9db1a1a55c18cd31faddd13dc709e45f19ac2891af4"}
Jul 21 17:08:34.270236 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.270203    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf479fb9fd631108086eb9db1a1a55c18cd31faddd13dc709e45f19ac2891af4"
Jul 21 17:08:34.411877 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.411857    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:34.494640 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.494608    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mghw2\" (UniqueName: \"kubernetes.io/projected/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-kube-api-access-mghw2\") pod \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") "
Jul 21 17:08:34.494826 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.494670    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-util\") pod \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") "
Jul 21 17:08:34.494826 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.494698    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-bundle\") pod \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\" (UID: \"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8\") "
Jul 21 17:08:34.495314 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.495288    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-bundle" (OuterVolumeSpecName: "bundle") pod "6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" (UID: "6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:34.496934 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.496907    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-kube-api-access-mghw2" (OuterVolumeSpecName: "kube-api-access-mghw2") pod "6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" (UID: "6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8"). InnerVolumeSpecName "kube-api-access-mghw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:08:34.500239 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.500210    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-util" (OuterVolumeSpecName: "util") pod "6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" (UID: "6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:34.595791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.595755    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:34.595791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.595782    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:34.595791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:34.595792    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mghw2\" (UniqueName: \"kubernetes.io/projected/6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8-kube-api-access-mghw2\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:35.277165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.277133    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl"
Jul 21 17:08:35.277165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.277158    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl" event={"ID":"6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8","Type":"ContainerDied","Data":"5432bc32a6d0fb6b84dc8272e32da7e39bd35abbed9737f5aa054e103d5ea11f"}
Jul 21 17:08:35.277647 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.277208    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5432bc32a6d0fb6b84dc8272e32da7e39bd35abbed9737f5aa054e103d5ea11f"
Jul 21 17:08:35.279510 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.279479    2561 generic.go:356] "Generic (PLEG): container finished" podID="8e055721-2549-419e-974e-bc59e120c7db" containerID="91001e5f22bf66c2c7ffae42e53968285597c23b5e751e680a68b161620b41d6" exitCode=0
Jul 21 17:08:35.279651 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.279581    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb" event={"ID":"8e055721-2549-419e-974e-bc59e120c7db","Type":"ContainerDied","Data":"91001e5f22bf66c2c7ffae42e53968285597c23b5e751e680a68b161620b41d6"}
Jul 21 17:08:35.413802 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.413776    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:35.501268 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.501238    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-bundle\") pod \"f59109e5-f725-4588-b6eb-fb632500afb5\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") "
Jul 21 17:08:35.501734 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.501713    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-bundle" (OuterVolumeSpecName: "bundle") pod "f59109e5-f725-4588-b6eb-fb632500afb5" (UID: "f59109e5-f725-4588-b6eb-fb632500afb5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:35.602301 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.602267    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lms26\" (UniqueName: \"kubernetes.io/projected/f59109e5-f725-4588-b6eb-fb632500afb5-kube-api-access-lms26\") pod \"f59109e5-f725-4588-b6eb-fb632500afb5\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") "
Jul 21 17:08:35.602301 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.602300    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-util\") pod \"f59109e5-f725-4588-b6eb-fb632500afb5\" (UID: \"f59109e5-f725-4588-b6eb-fb632500afb5\") "
Jul 21 17:08:35.602491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.602440    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:35.604530 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.604501    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f59109e5-f725-4588-b6eb-fb632500afb5-kube-api-access-lms26" (OuterVolumeSpecName: "kube-api-access-lms26") pod "f59109e5-f725-4588-b6eb-fb632500afb5" (UID: "f59109e5-f725-4588-b6eb-fb632500afb5"). InnerVolumeSpecName "kube-api-access-lms26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:08:35.607961 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.607934    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-util" (OuterVolumeSpecName: "util") pod "f59109e5-f725-4588-b6eb-fb632500afb5" (UID: "f59109e5-f725-4588-b6eb-fb632500afb5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:35.703414 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.703340    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lms26\" (UniqueName: \"kubernetes.io/projected/f59109e5-f725-4588-b6eb-fb632500afb5-kube-api-access-lms26\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:35.703414 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:35.703370    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f59109e5-f725-4588-b6eb-fb632500afb5-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:36.286715 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.286688    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b"
Jul 21 17:08:36.287098 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.286718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b" event={"ID":"f59109e5-f725-4588-b6eb-fb632500afb5","Type":"ContainerDied","Data":"1a8d1fe28a6c06b62a5395f03cbd8f3ff0b63560b04db84760669f0377b63e00"}
Jul 21 17:08:36.287098 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.286762    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a8d1fe28a6c06b62a5395f03cbd8f3ff0b63560b04db84760669f0377b63e00"
Jul 21 17:08:36.433973 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.433942    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:36.508678 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.508651    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-util\") pod \"8e055721-2549-419e-974e-bc59e120c7db\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") "
Jul 21 17:08:36.508865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.508689    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkplc\" (UniqueName: \"kubernetes.io/projected/8e055721-2549-419e-974e-bc59e120c7db-kube-api-access-kkplc\") pod \"8e055721-2549-419e-974e-bc59e120c7db\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") "
Jul 21 17:08:36.508865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.508709    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-bundle\") pod \"8e055721-2549-419e-974e-bc59e120c7db\" (UID: \"8e055721-2549-419e-974e-bc59e120c7db\") "
Jul 21 17:08:36.509199 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.509175    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-bundle" (OuterVolumeSpecName: "bundle") pod "8e055721-2549-419e-974e-bc59e120c7db" (UID: "8e055721-2549-419e-974e-bc59e120c7db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:36.511139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.511109    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e055721-2549-419e-974e-bc59e120c7db-kube-api-access-kkplc" (OuterVolumeSpecName: "kube-api-access-kkplc") pod "8e055721-2549-419e-974e-bc59e120c7db" (UID: "8e055721-2549-419e-974e-bc59e120c7db"). InnerVolumeSpecName "kube-api-access-kkplc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:08:36.515014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.514989    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-util" (OuterVolumeSpecName: "util") pod "8e055721-2549-419e-974e-bc59e120c7db" (UID: "8e055721-2549-419e-974e-bc59e120c7db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:08:36.609852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.609775    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkplc\" (UniqueName: \"kubernetes.io/projected/8e055721-2549-419e-974e-bc59e120c7db-kube-api-access-kkplc\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:36.609852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.609801    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:36.609852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:36.609814    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e055721-2549-419e-974e-bc59e120c7db-util\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:08:37.293930 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:37.293895    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb" event={"ID":"8e055721-2549-419e-974e-bc59e120c7db","Type":"ContainerDied","Data":"302dec33d620d997dc15656bbafb004e38835f4e4ce77d7a068c3b5fdec7204c"}
Jul 21 17:08:37.293930 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:37.293923    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb"
Jul 21 17:08:37.294356 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:37.293942    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="302dec33d620d997dc15656bbafb004e38835f4e4ce77d7a068c3b5fdec7204c"
Jul 21 17:08:39.341908 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.341865    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-567cb48fcd-2xknw"]
Jul 21 17:08:39.342360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342296    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="pull"
Jul 21 17:08:39.342360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342314    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="pull"
Jul 21 17:08:39.342360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342325    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="pull"
Jul 21 17:08:39.342360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342332    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="pull"
Jul 21 17:08:39.342360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342345    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="util"
Jul 21 17:08:39.342360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342353    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342374    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="pull"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342382    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="pull"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342394    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342401    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342413    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342421    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342431    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342438    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342451    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342459    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342468    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342477    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342489    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342497    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342509    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342517    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="util"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342525    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="pull"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342533    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="pull"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342629    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59109e5-f725-4588-b6eb-fb632500afb5" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342645    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e56816-ddf7-4baf-8dae-961e2b0ec61f" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342653    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e055721-2549-419e-974e-bc59e120c7db" containerName="extract"
Jul 21 17:08:39.342705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.342663    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8" containerName="extract"
Jul 21 17:08:39.346018 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.345998    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.356102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.356080    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567cb48fcd-2xknw"]
Jul 21 17:08:39.433855 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.433823    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-trusted-ca-bundle\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.433855 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.433860    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ggdz\" (UniqueName: \"kubernetes.io/projected/cd1b239a-9fd3-481a-a26f-2429019815f7-kube-api-access-7ggdz\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.434070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.433894    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-service-ca\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.434070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.433949    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1b239a-9fd3-481a-a26f-2429019815f7-console-oauth-config\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.434070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.433986    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-console-config\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.434181 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.434127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1b239a-9fd3-481a-a26f-2429019815f7-console-serving-cert\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.434181 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.434155    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-oauth-serving-cert\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535258 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-trusted-ca-bundle\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535258 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535256    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ggdz\" (UniqueName: \"kubernetes.io/projected/cd1b239a-9fd3-481a-a26f-2429019815f7-kube-api-access-7ggdz\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-service-ca\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1b239a-9fd3-481a-a26f-2429019815f7-console-oauth-config\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535385    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-console-config\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535640 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535627    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1b239a-9fd3-481a-a26f-2429019815f7-console-serving-cert\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.535692 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.535660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-oauth-serving-cert\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.536102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.536080    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-service-ca\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.536374 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.536124    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-console-config\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.536374 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.536228    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-trusted-ca-bundle\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.536374 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.536232    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1b239a-9fd3-481a-a26f-2429019815f7-oauth-serving-cert\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.538064 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.538042    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1b239a-9fd3-481a-a26f-2429019815f7-console-oauth-config\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.538175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.538126    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1b239a-9fd3-481a-a26f-2429019815f7-console-serving-cert\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.544867 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.544840    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ggdz\" (UniqueName: \"kubernetes.io/projected/cd1b239a-9fd3-481a-a26f-2429019815f7-kube-api-access-7ggdz\") pod \"console-567cb48fcd-2xknw\" (UID: \"cd1b239a-9fd3-481a-a26f-2429019815f7\") " pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.656073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.656036    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:39.793597 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:39.793570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567cb48fcd-2xknw"]
Jul 21 17:08:39.796224 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:39.796195    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1b239a_9fd3_481a_a26f_2429019815f7.slice/crio-6c8eb1a3af148b46c1097a8a836d9661c6c5e151644e6bb4b32bc0b3103d6d2c WatchSource:0}: Error finding container 6c8eb1a3af148b46c1097a8a836d9661c6c5e151644e6bb4b32bc0b3103d6d2c: Status 404 returned error can't find the container with id 6c8eb1a3af148b46c1097a8a836d9661c6c5e151644e6bb4b32bc0b3103d6d2c
Jul 21 17:08:40.311176 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:40.311146    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567cb48fcd-2xknw" event={"ID":"cd1b239a-9fd3-481a-a26f-2429019815f7","Type":"ContainerStarted","Data":"17c91df48f3cc987c9f3a06af9294f0b7e622d185e3e8059c7e428f3d1f86bd2"}
Jul 21 17:08:40.311343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:40.311196    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567cb48fcd-2xknw" event={"ID":"cd1b239a-9fd3-481a-a26f-2429019815f7","Type":"ContainerStarted","Data":"6c8eb1a3af148b46c1097a8a836d9661c6c5e151644e6bb4b32bc0b3103d6d2c"}
Jul 21 17:08:40.334529 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:40.334471    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-567cb48fcd-2xknw" podStartSLOduration=1.334452762 podStartE2EDuration="1.334452762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:40.332719345 +0000 UTC m=+400.297618950" watchObservedRunningTime="2026-07-21 17:08:40.334452762 +0000 UTC m=+400.299352368"
Jul 21 17:08:46.126713 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.126676    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"]
Jul 21 17:08:46.130176 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.130160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:08:46.132663 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.132640    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cpp88\""
Jul 21 17:08:46.145476 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.145451    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"]
Jul 21 17:08:46.189932 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.189906    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hflct\" (UniqueName: \"kubernetes.io/projected/944f38c9-7e23-4678-8d01-87aca07f4eac-kube-api-access-hflct\") pod \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" (UID: \"944f38c9-7e23-4678-8d01-87aca07f4eac\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:08:46.290811 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.290786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hflct\" (UniqueName: \"kubernetes.io/projected/944f38c9-7e23-4678-8d01-87aca07f4eac-kube-api-access-hflct\") pod \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" (UID: \"944f38c9-7e23-4678-8d01-87aca07f4eac\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:08:46.305023 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.304994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hflct\" (UniqueName: \"kubernetes.io/projected/944f38c9-7e23-4678-8d01-87aca07f4eac-kube-api-access-hflct\") pod \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" (UID: \"944f38c9-7e23-4678-8d01-87aca07f4eac\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:08:46.441043 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.441015    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:08:46.603628 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:46.603523    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"]
Jul 21 17:08:46.606678 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:46.606619    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944f38c9_7e23_4678_8d01_87aca07f4eac.slice/crio-780de2bdba9f5446560e84b40201c4204d0b288d8633f4be42994457ad44542f WatchSource:0}: Error finding container 780de2bdba9f5446560e84b40201c4204d0b288d8633f4be42994457ad44542f: Status 404 returned error can't find the container with id 780de2bdba9f5446560e84b40201c4204d0b288d8633f4be42994457ad44542f
Jul 21 17:08:47.348834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:47.348794    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk" event={"ID":"944f38c9-7e23-4678-8d01-87aca07f4eac","Type":"ContainerStarted","Data":"780de2bdba9f5446560e84b40201c4204d0b288d8633f4be42994457ad44542f"}
Jul 21 17:08:48.354502 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:48.354477    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk" event={"ID":"944f38c9-7e23-4678-8d01-87aca07f4eac","Type":"ContainerStarted","Data":"06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31"}
Jul 21 17:08:48.354891 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:48.354605    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:08:48.378854 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:48.378810    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk" podStartSLOduration=0.77509753 podStartE2EDuration="2.378795273s" totalImagesPullingTime="1.603697743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:48.376752774 +0000 UTC m=+408.341652382" watchObservedRunningTime="2026-07-21 17:08:48.378795273 +0000 UTC m=+408.343694877"
Jul 21 17:08:49.656602 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:49.656538    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:49.657046 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:49.656644    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:49.662222 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:49.662201    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:50.188528 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.188483    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"]
Jul 21 17:08:50.192442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.192420    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:08:50.194972 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.194951    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c2pmm\""
Jul 21 17:08:50.212950 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.212924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"]
Jul 21 17:08:50.227282 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.227261    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckgk4\" (UniqueName: \"kubernetes.io/projected/1a3265e6-7cf3-42e4-b485-53979720a590-kube-api-access-ckgk4\") pod \"authorino-operator-75cc8d5cb7-jmxcc\" (UID: \"1a3265e6-7cf3-42e4-b485-53979720a590\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:08:50.327650 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.327616    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckgk4\" (UniqueName: \"kubernetes.io/projected/1a3265e6-7cf3-42e4-b485-53979720a590-kube-api-access-ckgk4\") pod \"authorino-operator-75cc8d5cb7-jmxcc\" (UID: \"1a3265e6-7cf3-42e4-b485-53979720a590\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:08:50.338904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.338877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckgk4\" (UniqueName: \"kubernetes.io/projected/1a3265e6-7cf3-42e4-b485-53979720a590-kube-api-access-ckgk4\") pod \"authorino-operator-75cc8d5cb7-jmxcc\" (UID: \"1a3265e6-7cf3-42e4-b485-53979720a590\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:08:50.368487 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.368462    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-567cb48fcd-2xknw"
Jul 21 17:08:50.445254 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.445187    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85df7b475f-rtx2m"]
Jul 21 17:08:50.503356 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.503333    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:08:50.645009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:50.644982    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"]
Jul 21 17:08:50.646114 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:50.646092    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3265e6_7cf3_42e4_b485_53979720a590.slice/crio-2aa8f6ac6b6d41150068cb4c178b80c2c5feb33b234c14d40685d9e695fd11d0 WatchSource:0}: Error finding container 2aa8f6ac6b6d41150068cb4c178b80c2c5feb33b234c14d40685d9e695fd11d0: Status 404 returned error can't find the container with id 2aa8f6ac6b6d41150068cb4c178b80c2c5feb33b234c14d40685d9e695fd11d0
Jul 21 17:08:51.371885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:51.371850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc" event={"ID":"1a3265e6-7cf3-42e4-b485-53979720a590","Type":"ContainerStarted","Data":"2aa8f6ac6b6d41150068cb4c178b80c2c5feb33b234c14d40685d9e695fd11d0"}
Jul 21 17:08:53.384420 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:53.384390    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc" event={"ID":"1a3265e6-7cf3-42e4-b485-53979720a590","Type":"ContainerStarted","Data":"a04b6c4513e418ef9fc5624f7e1167c5120b3efc3eb1cfdb17b9020e80d13d88"}
Jul 21 17:08:53.384834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:53.384675    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:08:55.204360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.204300    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc" podStartSLOduration=3.003305646 podStartE2EDuration="5.204280487s" totalImagesPullingTime="2.200974841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:53.447860755 +0000 UTC m=+413.412760359" watchObservedRunningTime="2026-07-21 17:08:55.204280487 +0000 UTC m=+415.169180091"
Jul 21 17:08:55.206194 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.206170    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"]
Jul 21 17:08:55.208852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.208830    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.212473 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.212455    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bc4hz\""
Jul 21 17:08:55.225232 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.225206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"]
Jul 21 17:08:55.275612 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.275552    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bd90c3e-4a99-4e01-9408-7df4a065fd90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.275749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.275675    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdm9n\" (UniqueName: \"kubernetes.io/projected/4bd90c3e-4a99-4e01-9408-7df4a065fd90-kube-api-access-tdm9n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.376763 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.376728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bd90c3e-4a99-4e01-9408-7df4a065fd90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.376902 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.376789    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdm9n\" (UniqueName: \"kubernetes.io/projected/4bd90c3e-4a99-4e01-9408-7df4a065fd90-kube-api-access-tdm9n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.377157 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.377137    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bd90c3e-4a99-4e01-9408-7df4a065fd90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.388846 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.388815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdm9n\" (UniqueName: \"kubernetes.io/projected/4bd90c3e-4a99-4e01-9408-7df4a065fd90-kube-api-access-tdm9n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.519471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.519388    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:08:55.668346 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:55.668241    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"]
Jul 21 17:08:55.671271 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:08:55.671238    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd90c3e_4a99_4e01_9408_7df4a065fd90.slice/crio-8ea4c9e25bbccdf89a0817422a121ea122e382864d7dab0f30b2910bfc5c672a WatchSource:0}: Error finding container 8ea4c9e25bbccdf89a0817422a121ea122e382864d7dab0f30b2910bfc5c672a: Status 404 returned error can't find the container with id 8ea4c9e25bbccdf89a0817422a121ea122e382864d7dab0f30b2910bfc5c672a
Jul 21 17:08:56.400889 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:56.400854    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f" event={"ID":"4bd90c3e-4a99-4e01-9408-7df4a065fd90","Type":"ContainerStarted","Data":"8ea4c9e25bbccdf89a0817422a121ea122e382864d7dab0f30b2910bfc5c672a"}
Jul 21 17:08:59.362419 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:08:59.362379    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:03.440412 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:03.440370    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f" event={"ID":"4bd90c3e-4a99-4e01-9408-7df4a065fd90","Type":"ContainerStarted","Data":"9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d"}
Jul 21 17:09:03.440974 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:03.440461    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:03.481436 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:03.481380    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f" podStartSLOduration=1.6448732449999999 podStartE2EDuration="8.481358914s" totalImagesPullingTime="6.836485669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:03.478825089 +0000 UTC m=+423.443724694" watchObservedRunningTime="2026-07-21 17:09:03.481358914 +0000 UTC m=+423.446258517"
Jul 21 17:09:04.391840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:04.391802    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmxcc"
Jul 21 17:09:14.448783 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:14.448750    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:15.466445 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.466406    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85df7b475f-rtx2m" podUID="18c91dbb-be61-4edf-b6f7-8a676e181a15" containerName="console" containerID="cri-o://0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836" gracePeriod=15
Jul 21 17:09:15.705069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.705045    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:09:15.856784 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856701    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-config\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.856784 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856738    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8c95\" (UniqueName: \"kubernetes.io/projected/18c91dbb-be61-4edf-b6f7-8a676e181a15-kube-api-access-h8c95\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.856784 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856767    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-serving-cert\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.857059 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856793    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-oauth-config\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.857059 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856821    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-oauth-serving-cert\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.857059 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856843    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-service-ca\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.857059 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.856974    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-trusted-ca-bundle\") pod \"18c91dbb-be61-4edf-b6f7-8a676e181a15\" (UID: \"18c91dbb-be61-4edf-b6f7-8a676e181a15\") "
Jul 21 17:09:15.857275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857227    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-config" (OuterVolumeSpecName: "console-config") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:09:15.857275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857230    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-service-ca" (OuterVolumeSpecName: "service-ca") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:09:15.857275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857266    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:09:15.857677 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857371    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-config\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:15.857787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857692    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-oauth-serving-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:15.857787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857713    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:15.857787 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.857605    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:09:15.859040 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.859020    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:09:15.859207 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.859187    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c91dbb-be61-4edf-b6f7-8a676e181a15-kube-api-access-h8c95" (OuterVolumeSpecName: "kube-api-access-h8c95") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "kube-api-access-h8c95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:09:15.859270 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.859186    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "18c91dbb-be61-4edf-b6f7-8a676e181a15" (UID: "18c91dbb-be61-4edf-b6f7-8a676e181a15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:09:15.958428 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.958393    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8c95\" (UniqueName: \"kubernetes.io/projected/18c91dbb-be61-4edf-b6f7-8a676e181a15-kube-api-access-h8c95\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:15.958428 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.958422    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-serving-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:15.958428 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.958434    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c91dbb-be61-4edf-b6f7-8a676e181a15-console-oauth-config\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:15.958710 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:15.958444    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c91dbb-be61-4edf-b6f7-8a676e181a15-trusted-ca-bundle\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:16.109772 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.109644    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"]
Jul 21 17:09:16.109952 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.109926    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" containerName="manager" containerID="cri-o://9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d" gracePeriod=2
Jul 21 17:09:16.112158 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.112121    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.114207 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.114164    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"]
Jul 21 17:09:16.130187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.130166    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"]
Jul 21 17:09:16.130397 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.130355    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" containerName="manager" containerID="cri-o://06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31" gracePeriod=2
Jul 21 17:09:16.132666 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.132638    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.138026 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138002    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"]
Jul 21 17:09:16.138488 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138469    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18c91dbb-be61-4edf-b6f7-8a676e181a15" containerName="console"
Jul 21 17:09:16.138539 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138493    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c91dbb-be61-4edf-b6f7-8a676e181a15" containerName="console"
Jul 21 17:09:16.138539 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138519    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" containerName="manager"
Jul 21 17:09:16.138539 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138528    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" containerName="manager"
Jul 21 17:09:16.138694 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138679    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" containerName="manager"
Jul 21 17:09:16.138729 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.138704    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="18c91dbb-be61-4edf-b6f7-8a676e181a15" containerName="console"
Jul 21 17:09:16.141892 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.141873    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.149102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.149076    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"]
Jul 21 17:09:16.157692 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.157668    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"]
Jul 21 17:09:16.158038 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.157934    2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"944f38c9-7e23-4678-8d01-87aca07f4eac\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-sqwgk\": pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.160638 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.160605    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"]
Jul 21 17:09:16.160745 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.160658    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.160988 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.160970    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" containerName="manager"
Jul 21 17:09:16.160988 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.160988    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" containerName="manager"
Jul 21 17:09:16.161123 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.161040    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" containerName="manager"
Jul 21 17:09:16.163886 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.163869    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:16.176888 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.176865    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"]
Jul 21 17:09:16.194583 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.193836    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.214790 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.214758    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.216686 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.216664    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.260974 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.260929    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7cql\" (UniqueName: \"kubernetes.io/projected/b74262ec-7b72-4a58-a673-0316ebfadf0f-kube-api-access-t7cql\") pod \"limitador-operator-controller-manager-859d7b78bd-fktzx\" (UID: \"b74262ec-7b72-4a58-a673-0316ebfadf0f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:16.261093 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.260984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a67d1dac-0dcc-4b46-bf47-a94deec6544a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qmmdl\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.261093 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.261075    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwxf8\" (UniqueName: \"kubernetes.io/projected/a67d1dac-0dcc-4b46-bf47-a94deec6544a-kube-api-access-fwxf8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qmmdl\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.362453 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.362422    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7cql\" (UniqueName: \"kubernetes.io/projected/b74262ec-7b72-4a58-a673-0316ebfadf0f-kube-api-access-t7cql\") pod \"limitador-operator-controller-manager-859d7b78bd-fktzx\" (UID: \"b74262ec-7b72-4a58-a673-0316ebfadf0f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:16.362614 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.362461    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a67d1dac-0dcc-4b46-bf47-a94deec6544a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qmmdl\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.362614 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.362516    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwxf8\" (UniqueName: \"kubernetes.io/projected/a67d1dac-0dcc-4b46-bf47-a94deec6544a-kube-api-access-fwxf8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qmmdl\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.362959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.362939    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a67d1dac-0dcc-4b46-bf47-a94deec6544a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qmmdl\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.368979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.368953    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.372846 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.372824    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.373832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.373808    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.374144 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.374126    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7cql\" (UniqueName: \"kubernetes.io/projected/b74262ec-7b72-4a58-a673-0316ebfadf0f-kube-api-access-t7cql\") pod \"limitador-operator-controller-manager-859d7b78bd-fktzx\" (UID: \"b74262ec-7b72-4a58-a673-0316ebfadf0f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:16.375003 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.374982    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.376741 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.376714    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.377781 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.377765    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwxf8\" (UniqueName: \"kubernetes.io/projected/a67d1dac-0dcc-4b46-bf47-a94deec6544a-kube-api-access-fwxf8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qmmdl\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.378523 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.378495    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.462820 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.462791    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hflct\" (UniqueName: \"kubernetes.io/projected/944f38c9-7e23-4678-8d01-87aca07f4eac-kube-api-access-hflct\") pod \"944f38c9-7e23-4678-8d01-87aca07f4eac\" (UID: \"944f38c9-7e23-4678-8d01-87aca07f4eac\") "
Jul 21 17:09:16.462980 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.462880    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdm9n\" (UniqueName: \"kubernetes.io/projected/4bd90c3e-4a99-4e01-9408-7df4a065fd90-kube-api-access-tdm9n\") pod \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") "
Jul 21 17:09:16.462980 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.462933    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bd90c3e-4a99-4e01-9408-7df4a065fd90-extensions-socket-volume\") pod \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\" (UID: \"4bd90c3e-4a99-4e01-9408-7df4a065fd90\") "
Jul 21 17:09:16.463401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.463379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bd90c3e-4a99-4e01-9408-7df4a065fd90-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4bd90c3e-4a99-4e01-9408-7df4a065fd90" (UID: "4bd90c3e-4a99-4e01-9408-7df4a065fd90"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:09:16.465027 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.464990    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bd90c3e-4a99-4e01-9408-7df4a065fd90-kube-api-access-tdm9n" (OuterVolumeSpecName: "kube-api-access-tdm9n") pod "4bd90c3e-4a99-4e01-9408-7df4a065fd90" (UID: "4bd90c3e-4a99-4e01-9408-7df4a065fd90"). InnerVolumeSpecName "kube-api-access-tdm9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:09:16.465027 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.465007    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/944f38c9-7e23-4678-8d01-87aca07f4eac-kube-api-access-hflct" (OuterVolumeSpecName: "kube-api-access-hflct") pod "944f38c9-7e23-4678-8d01-87aca07f4eac" (UID: "944f38c9-7e23-4678-8d01-87aca07f4eac"). InnerVolumeSpecName "kube-api-access-hflct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:09:16.507266 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.507230    2561 generic.go:356] "Generic (PLEG): container finished" podID="944f38c9-7e23-4678-8d01-87aca07f4eac" containerID="06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31" exitCode=0
Jul 21 17:09:16.507682 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.507304    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.508836 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.508814    2561 generic.go:356] "Generic (PLEG): container finished" podID="18c91dbb-be61-4edf-b6f7-8a676e181a15" containerID="0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836" exitCode=0
Jul 21 17:09:16.508961 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.508935    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85df7b475f-rtx2m"
Jul 21 17:09:16.509962 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.509931    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.510730 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.510701    2561 generic.go:356] "Generic (PLEG): container finished" podID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" containerID="9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d" exitCode=0
Jul 21 17:09:16.510834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.510817    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.512618 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.512589    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.513725 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.513707    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" path="/var/lib/kubelet/pods/4bd90c3e-4a99-4e01-9408-7df4a065fd90/volumes"
Jul 21 17:09:16.514045 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.514031    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" path="/var/lib/kubelet/pods/944f38c9-7e23-4678-8d01-87aca07f4eac/volumes"
Jul 21 17:09:16.514498 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.514480    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85df7b475f-rtx2m" event={"ID":"18c91dbb-be61-4edf-b6f7-8a676e181a15","Type":"ContainerDied","Data":"0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836"}
Jul 21 17:09:16.514593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.514518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85df7b475f-rtx2m" event={"ID":"18c91dbb-be61-4edf-b6f7-8a676e181a15","Type":"ContainerDied","Data":"ecaf922b435f04e8e408437249c160a057638b800e816f4ebef08f01b2556786"}
Jul 21 17:09:16.514593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.514570    2561 scope.go:117] "RemoveContainer" containerID="06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31"
Jul 21 17:09:16.522317 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.522280    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rj66f\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="4bd90c3e-4a99-4e01-9408-7df4a065fd90" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rj66f"
Jul 21 17:09:16.524304 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.524276    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sqwgk\" is forbidden: User \"system:node:ip-10-0-140-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-24.ec2.internal' and this object" podUID="944f38c9-7e23-4678-8d01-87aca07f4eac" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqwgk"
Jul 21 17:09:16.526908 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.526885    2561 scope.go:117] "RemoveContainer" containerID="06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31"
Jul 21 17:09:16.527191 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.527173    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31\": container with ID starting with 06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31 not found: ID does not exist" containerID="06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31"
Jul 21 17:09:16.527257 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.527198    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31"} err="failed to get container status \"06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31\": rpc error: code = NotFound desc = could not find container \"06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31\": container with ID starting with 06798e551e8ef3bfe5cb94adbf0fd2be3e6c3cdc76113011d4c2a390b3171a31 not found: ID does not exist"
Jul 21 17:09:16.527257 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.527214    2561 scope.go:117] "RemoveContainer" containerID="0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836"
Jul 21 17:09:16.535060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.535044    2561 scope.go:117] "RemoveContainer" containerID="0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836"
Jul 21 17:09:16.535302 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.535284    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836\": container with ID starting with 0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836 not found: ID does not exist" containerID="0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836"
Jul 21 17:09:16.535350 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.535311    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836"} err="failed to get container status \"0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836\": rpc error: code = NotFound desc = could not find container \"0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836\": container with ID starting with 0dfe9fcaef0a1ba565d451c9eff730d105b645c451a1b5cd5af96a88d040d836 not found: ID does not exist"
Jul 21 17:09:16.535350 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.535328    2561 scope.go:117] "RemoveContainer" containerID="9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d"
Jul 21 17:09:16.542066 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.541898    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:16.543144 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.543129    2561 scope.go:117] "RemoveContainer" containerID="9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d"
Jul 21 17:09:16.543393 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:16.543376    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d\": container with ID starting with 9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d not found: ID does not exist" containerID="9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d"
Jul 21 17:09:16.543446 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.543400    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d"} err="failed to get container status \"9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d\": rpc error: code = NotFound desc = could not find container \"9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d\": container with ID starting with 9afe8847650b53aaa469cbaa145aac84a36dbde3e8c389f969d733e05c90e12d not found: ID does not exist"
Jul 21 17:09:16.544689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.544671    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85df7b475f-rtx2m"]
Jul 21 17:09:16.548406 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.548386    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:16.553623 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.553606    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85df7b475f-rtx2m"]
Jul 21 17:09:16.564303 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.564284    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bd90c3e-4a99-4e01-9408-7df4a065fd90-extensions-socket-volume\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:16.564303 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.564306    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hflct\" (UniqueName: \"kubernetes.io/projected/944f38c9-7e23-4678-8d01-87aca07f4eac-kube-api-access-hflct\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:16.564459 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.564320    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdm9n\" (UniqueName: \"kubernetes.io/projected/4bd90c3e-4a99-4e01-9408-7df4a065fd90-kube-api-access-tdm9n\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:16.698920 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.698894    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"]
Jul 21 17:09:16.700604 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:09:16.700545    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67d1dac_0dcc_4b46_bf47_a94deec6544a.slice/crio-97727dbf4e877619b35a18574fae3bae49533528304e7c02e443744e3290f26a WatchSource:0}: Error finding container 97727dbf4e877619b35a18574fae3bae49533528304e7c02e443744e3290f26a: Status 404 returned error can't find the container with id 97727dbf4e877619b35a18574fae3bae49533528304e7c02e443744e3290f26a
Jul 21 17:09:16.715925 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:16.715906    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"]
Jul 21 17:09:16.718215 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:09:16.718195    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74262ec_7b72_4a58_a673_0316ebfadf0f.slice/crio-be457dffd963deb6887383a1c3ff82c8db06232f8643d67827e0a57651593d64 WatchSource:0}: Error finding container be457dffd963deb6887383a1c3ff82c8db06232f8643d67827e0a57651593d64: Status 404 returned error can't find the container with id be457dffd963deb6887383a1c3ff82c8db06232f8643d67827e0a57651593d64
Jul 21 17:09:17.517810 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.517772    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl" event={"ID":"a67d1dac-0dcc-4b46-bf47-a94deec6544a","Type":"ContainerStarted","Data":"df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b"}
Jul 21 17:09:17.518241 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.517832    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl" event={"ID":"a67d1dac-0dcc-4b46-bf47-a94deec6544a","Type":"ContainerStarted","Data":"97727dbf4e877619b35a18574fae3bae49533528304e7c02e443744e3290f26a"}
Jul 21 17:09:17.518241 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.517975    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:17.519291 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.519264    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx" event={"ID":"b74262ec-7b72-4a58-a673-0316ebfadf0f","Type":"ContainerStarted","Data":"aed9b7c83aed5bfeb66d75d197fb6fb07b6f96008fcfc840f4d7389522de199f"}
Jul 21 17:09:17.519400 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.519314    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx" event={"ID":"b74262ec-7b72-4a58-a673-0316ebfadf0f","Type":"ContainerStarted","Data":"be457dffd963deb6887383a1c3ff82c8db06232f8643d67827e0a57651593d64"}
Jul 21 17:09:17.519400 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.519368    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:17.543590 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.543528    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl" podStartSLOduration=1.5435119400000001 podStartE2EDuration="1.54351194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:17.541337912 +0000 UTC m=+437.506237523" watchObservedRunningTime="2026-07-21 17:09:17.54351194 +0000 UTC m=+437.508411543"
Jul 21 17:09:17.563870 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:17.563825    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx" podStartSLOduration=1.563810449 podStartE2EDuration="1.563810449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:17.562709786 +0000 UTC m=+437.527609382" watchObservedRunningTime="2026-07-21 17:09:17.563810449 +0000 UTC m=+437.528710052"
Jul 21 17:09:18.513965 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:18.513925    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18c91dbb-be61-4edf-b6f7-8a676e181a15" path="/var/lib/kubelet/pods/18c91dbb-be61-4edf-b6f7-8a676e181a15/volumes"
Jul 21 17:09:28.527932 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:28.527851    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fktzx"
Jul 21 17:09:28.528276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:28.527933    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:32.337615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.337574    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"]
Jul 21 17:09:32.338081 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.337831    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl" podUID="a67d1dac-0dcc-4b46-bf47-a94deec6544a" containerName="manager" containerID="cri-o://df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b" gracePeriod=10
Jul 21 17:09:32.579164 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.579140    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:32.584534 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.584512    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwxf8\" (UniqueName: \"kubernetes.io/projected/a67d1dac-0dcc-4b46-bf47-a94deec6544a-kube-api-access-fwxf8\") pod \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") "
Jul 21 17:09:32.584655 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.584626    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a67d1dac-0dcc-4b46-bf47-a94deec6544a-extensions-socket-volume\") pod \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\" (UID: \"a67d1dac-0dcc-4b46-bf47-a94deec6544a\") "
Jul 21 17:09:32.585092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.585068    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67d1dac-0dcc-4b46-bf47-a94deec6544a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a67d1dac-0dcc-4b46-bf47-a94deec6544a" (UID: "a67d1dac-0dcc-4b46-bf47-a94deec6544a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:09:32.586730 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.586708    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a67d1dac-0dcc-4b46-bf47-a94deec6544a-kube-api-access-fwxf8" (OuterVolumeSpecName: "kube-api-access-fwxf8") pod "a67d1dac-0dcc-4b46-bf47-a94deec6544a" (UID: "a67d1dac-0dcc-4b46-bf47-a94deec6544a"). InnerVolumeSpecName "kube-api-access-fwxf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:09:32.595348 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.595328    2561 generic.go:356] "Generic (PLEG): container finished" podID="a67d1dac-0dcc-4b46-bf47-a94deec6544a" containerID="df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b" exitCode=0
Jul 21 17:09:32.595444 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.595371    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"
Jul 21 17:09:32.595444 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.595411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl" event={"ID":"a67d1dac-0dcc-4b46-bf47-a94deec6544a","Type":"ContainerDied","Data":"df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b"}
Jul 21 17:09:32.595521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.595462    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl" event={"ID":"a67d1dac-0dcc-4b46-bf47-a94deec6544a","Type":"ContainerDied","Data":"97727dbf4e877619b35a18574fae3bae49533528304e7c02e443744e3290f26a"}
Jul 21 17:09:32.595521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.595484    2561 scope.go:117] "RemoveContainer" containerID="df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b"
Jul 21 17:09:32.605284 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.605267    2561 scope.go:117] "RemoveContainer" containerID="df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b"
Jul 21 17:09:32.605524 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:09:32.605509    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b\": container with ID starting with df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b not found: ID does not exist" containerID="df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b"
Jul 21 17:09:32.605622 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.605530    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b"} err="failed to get container status \"df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b\": rpc error: code = NotFound desc = could not find container \"df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b\": container with ID starting with df5b93912d1684036a81fed3d431d39f6a2350ccd0275871514e7b03d459fa8b not found: ID does not exist"
Jul 21 17:09:32.623345 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.623316    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"]
Jul 21 17:09:32.623667 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.623654    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67d1dac-0dcc-4b46-bf47-a94deec6544a" containerName="manager"
Jul 21 17:09:32.623719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.623669    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67d1dac-0dcc-4b46-bf47-a94deec6544a" containerName="manager"
Jul 21 17:09:32.623761 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.623730    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a67d1dac-0dcc-4b46-bf47-a94deec6544a" containerName="manager"
Jul 21 17:09:32.628067 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.628050    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"]
Jul 21 17:09:32.628149 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.628123    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.631130 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.631108    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bc4hz\""
Jul 21 17:09:32.634911 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.634888    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qmmdl"]
Jul 21 17:09:32.647723 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.647656    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"]
Jul 21 17:09:32.685138 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.685103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsfld\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.685281 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.685144    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb85q\" (UniqueName: \"kubernetes.io/projected/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-kube-api-access-tb85q\") pod \"kuadrant-operator-controller-manager-86878c769f-rsfld\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.685281 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.685208    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a67d1dac-0dcc-4b46-bf47-a94deec6544a-extensions-socket-volume\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:32.685281 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.685219    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwxf8\" (UniqueName: \"kubernetes.io/projected/a67d1dac-0dcc-4b46-bf47-a94deec6544a-kube-api-access-fwxf8\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:09:32.786459 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.786417    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsfld\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.786632 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.786465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb85q\" (UniqueName: \"kubernetes.io/projected/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-kube-api-access-tb85q\") pod \"kuadrant-operator-controller-manager-86878c769f-rsfld\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.786911 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.786891    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsfld\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.796200 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.796167    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb85q\" (UniqueName: \"kubernetes.io/projected/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-kube-api-access-tb85q\") pod \"kuadrant-operator-controller-manager-86878c769f-rsfld\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:32.939007 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:32.938974    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:33.082856 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:33.082833    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"]
Jul 21 17:09:33.083919 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:09:33.083891    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc76014de_0db5_4f2f_a3ba_bd3eaadc1742.slice/crio-78bba8452f320e818e255eb1c3d384c2b83a7c0936c05e9369e93a8abe85a245 WatchSource:0}: Error finding container 78bba8452f320e818e255eb1c3d384c2b83a7c0936c05e9369e93a8abe85a245: Status 404 returned error can't find the container with id 78bba8452f320e818e255eb1c3d384c2b83a7c0936c05e9369e93a8abe85a245
Jul 21 17:09:33.604019 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:33.603983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld" event={"ID":"c76014de-0db5-4f2f-a3ba-bd3eaadc1742","Type":"ContainerStarted","Data":"5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa"}
Jul 21 17:09:33.604405 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:33.604040    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld" event={"ID":"c76014de-0db5-4f2f-a3ba-bd3eaadc1742","Type":"ContainerStarted","Data":"78bba8452f320e818e255eb1c3d384c2b83a7c0936c05e9369e93a8abe85a245"}
Jul 21 17:09:33.604405 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:33.604099    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:09:33.631789 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:33.631748    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld" podStartSLOduration=1.631732688 podStartE2EDuration="1.631732688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:33.629818519 +0000 UTC m=+453.594718127" watchObservedRunningTime="2026-07-21 17:09:33.631732688 +0000 UTC m=+453.596632292"
Jul 21 17:09:34.516587 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:34.513614    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a67d1dac-0dcc-4b46-bf47-a94deec6544a" path="/var/lib/kubelet/pods/a67d1dac-0dcc-4b46-bf47-a94deec6544a/volumes"
Jul 21 17:09:44.611462 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:09:44.611432    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:10:05.524636 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.524598    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-gl956"]
Jul 21 17:10:05.527429 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.527412    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:05.530205 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.530181    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7qs8z\""
Jul 21 17:10:05.538208 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.538181    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gl956"]
Jul 21 17:10:05.666913 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.666879    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6v6\" (UniqueName: \"kubernetes.io/projected/18d5da59-606e-4c45-9dae-1a56aba41f03-kube-api-access-9t6v6\") pod \"authorino-75d556bd77-gl956\" (UID: \"18d5da59-606e-4c45-9dae-1a56aba41f03\") " pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:05.767699 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.767665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6v6\" (UniqueName: \"kubernetes.io/projected/18d5da59-606e-4c45-9dae-1a56aba41f03-kube-api-access-9t6v6\") pod \"authorino-75d556bd77-gl956\" (UID: \"18d5da59-606e-4c45-9dae-1a56aba41f03\") " pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:05.777692 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.777622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6v6\" (UniqueName: \"kubernetes.io/projected/18d5da59-606e-4c45-9dae-1a56aba41f03-kube-api-access-9t6v6\") pod \"authorino-75d556bd77-gl956\" (UID: \"18d5da59-606e-4c45-9dae-1a56aba41f03\") " pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:05.837178 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.837147    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:05.966070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:05.966044    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gl956"]
Jul 21 17:10:05.968439 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:10:05.968403    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18d5da59_606e_4c45_9dae_1a56aba41f03.slice/crio-709ad3a698e26ff91a78291c1c3914683bc1ed2e46b91c3bad8413b0e5467619 WatchSource:0}: Error finding container 709ad3a698e26ff91a78291c1c3914683bc1ed2e46b91c3bad8413b0e5467619: Status 404 returned error can't find the container with id 709ad3a698e26ff91a78291c1c3914683bc1ed2e46b91c3bad8413b0e5467619
Jul 21 17:10:06.785451 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:06.785379    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gl956" event={"ID":"18d5da59-606e-4c45-9dae-1a56aba41f03","Type":"ContainerStarted","Data":"709ad3a698e26ff91a78291c1c3914683bc1ed2e46b91c3bad8413b0e5467619"}
Jul 21 17:10:09.805213 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:09.805176    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gl956" event={"ID":"18d5da59-606e-4c45-9dae-1a56aba41f03","Type":"ContainerStarted","Data":"e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da"}
Jul 21 17:10:09.826293 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:09.826236    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-gl956" podStartSLOduration=1.927464655 podStartE2EDuration="4.826222307s" totalImagesPullingTime="2.898757652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:10:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:10:09.823352082 +0000 UTC m=+489.788251686" watchObservedRunningTime="2026-07-21 17:10:09.826222307 +0000 UTC m=+489.791121911"
Jul 21 17:10:40.576882 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.576848    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6494d5c85b-45klz"]
Jul 21 17:10:40.581064 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.581043    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.583427 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.583409    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 21 17:10:40.589261 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.589240    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6494d5c85b-45klz"]
Jul 21 17:10:40.659721 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.659700    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6trtv\" (UniqueName: \"kubernetes.io/projected/e36f22cf-a567-4d65-8630-ab214debcecf-kube-api-access-6trtv\") pod \"authorino-6494d5c85b-45klz\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") " pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.659845 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.659755    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e36f22cf-a567-4d65-8630-ab214debcecf-tls-cert\") pod \"authorino-6494d5c85b-45klz\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") " pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.760271 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.760240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6trtv\" (UniqueName: \"kubernetes.io/projected/e36f22cf-a567-4d65-8630-ab214debcecf-kube-api-access-6trtv\") pod \"authorino-6494d5c85b-45klz\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") " pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.760398 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.760298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e36f22cf-a567-4d65-8630-ab214debcecf-tls-cert\") pod \"authorino-6494d5c85b-45klz\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") " pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.762635 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.762613    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e36f22cf-a567-4d65-8630-ab214debcecf-tls-cert\") pod \"authorino-6494d5c85b-45klz\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") " pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.769549 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.769526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6trtv\" (UniqueName: \"kubernetes.io/projected/e36f22cf-a567-4d65-8630-ab214debcecf-kube-api-access-6trtv\") pod \"authorino-6494d5c85b-45klz\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") " pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:40.890791 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:40.890736    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:10:41.026069 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:41.026022    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6494d5c85b-45klz"]
Jul 21 17:10:41.966141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:41.966097    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6494d5c85b-45klz" event={"ID":"e36f22cf-a567-4d65-8630-ab214debcecf","Type":"ContainerStarted","Data":"802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc"}
Jul 21 17:10:41.966536 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:41.966148    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6494d5c85b-45klz" event={"ID":"e36f22cf-a567-4d65-8630-ab214debcecf","Type":"ContainerStarted","Data":"68ab535383c52d06b70bbbaf1d9f051f15e05572fd752c5b8b3b63cb196e6c1c"}
Jul 21 17:10:41.984906 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:41.984858    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6494d5c85b-45klz" podStartSLOduration=1.351017985 podStartE2EDuration="1.984843319s" totalImagesPullingTime="633.825334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:10:41.982994891 +0000 UTC m=+521.947894495" watchObservedRunningTime="2026-07-21 17:10:41.984843319 +0000 UTC m=+521.949742923"
Jul 21 17:10:42.016524 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.016487    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gl956"]
Jul 21 17:10:42.016828 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.016796    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-gl956" podUID="18d5da59-606e-4c45-9dae-1a56aba41f03" containerName="authorino" containerID="cri-o://e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da" gracePeriod=30
Jul 21 17:10:42.257582 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.257542    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:42.373826 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.373791    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t6v6\" (UniqueName: \"kubernetes.io/projected/18d5da59-606e-4c45-9dae-1a56aba41f03-kube-api-access-9t6v6\") pod \"18d5da59-606e-4c45-9dae-1a56aba41f03\" (UID: \"18d5da59-606e-4c45-9dae-1a56aba41f03\") "
Jul 21 17:10:42.376020 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.375997    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d5da59-606e-4c45-9dae-1a56aba41f03-kube-api-access-9t6v6" (OuterVolumeSpecName: "kube-api-access-9t6v6") pod "18d5da59-606e-4c45-9dae-1a56aba41f03" (UID: "18d5da59-606e-4c45-9dae-1a56aba41f03"). InnerVolumeSpecName "kube-api-access-9t6v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:10:42.474515 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.474481    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t6v6\" (UniqueName: \"kubernetes.io/projected/18d5da59-606e-4c45-9dae-1a56aba41f03-kube-api-access-9t6v6\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:10:42.972192 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.972154    2561 generic.go:356] "Generic (PLEG): container finished" podID="18d5da59-606e-4c45-9dae-1a56aba41f03" containerID="e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da" exitCode=0
Jul 21 17:10:42.972662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.972197    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gl956"
Jul 21 17:10:42.972662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.972246    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gl956" event={"ID":"18d5da59-606e-4c45-9dae-1a56aba41f03","Type":"ContainerDied","Data":"e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da"}
Jul 21 17:10:42.972662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.972285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gl956" event={"ID":"18d5da59-606e-4c45-9dae-1a56aba41f03","Type":"ContainerDied","Data":"709ad3a698e26ff91a78291c1c3914683bc1ed2e46b91c3bad8413b0e5467619"}
Jul 21 17:10:42.972662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.972307    2561 scope.go:117] "RemoveContainer" containerID="e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da"
Jul 21 17:10:42.981159 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.981143    2561 scope.go:117] "RemoveContainer" containerID="e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da"
Jul 21 17:10:42.981407 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:10:42.981390    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da\": container with ID starting with e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da not found: ID does not exist" containerID="e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da"
Jul 21 17:10:42.981450 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.981414    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da"} err="failed to get container status \"e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da\": rpc error: code = NotFound desc = could not find container \"e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da\": container with ID starting with e5335bc8499f640013baa5f2e20ad0b17234a465209429d68db3441d752ed4da not found: ID does not exist"
Jul 21 17:10:42.992207 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.992184    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gl956"]
Jul 21 17:10:42.996276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:42.996258    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gl956"]
Jul 21 17:10:44.512413 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:44.512383    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d5da59-606e-4c45-9dae-1a56aba41f03" path="/var/lib/kubelet/pods/18d5da59-606e-4c45-9dae-1a56aba41f03/volumes"
Jul 21 17:10:46.560061 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.560029    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7955c6964b-742pj"]
Jul 21 17:10:46.560487 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.560338    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d5da59-606e-4c45-9dae-1a56aba41f03" containerName="authorino"
Jul 21 17:10:46.560487 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.560348    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d5da59-606e-4c45-9dae-1a56aba41f03" containerName="authorino"
Jul 21 17:10:46.560487 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.560399    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d5da59-606e-4c45-9dae-1a56aba41f03" containerName="authorino"
Jul 21 17:10:46.565108 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.565088    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.568522 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.568501    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 17:10:46.568643 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.568527    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rk7d7\""
Jul 21 17:10:46.568643 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.568497    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 17:10:46.574322 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.574301    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955c6964b-742pj"]
Jul 21 17:10:46.711800 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.711762    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k6gm\" (UniqueName: \"kubernetes.io/projected/7cfac97c-292e-4b27-aeb7-acad5b69564f-kube-api-access-4k6gm\") pod \"maas-controller-7955c6964b-742pj\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") " pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.711800 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.711806    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfac97c-292e-4b27-aeb7-acad5b69564f-webhook-cert\") pod \"maas-controller-7955c6964b-742pj\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") " pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.813293 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.813213    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k6gm\" (UniqueName: \"kubernetes.io/projected/7cfac97c-292e-4b27-aeb7-acad5b69564f-kube-api-access-4k6gm\") pod \"maas-controller-7955c6964b-742pj\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") " pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.813293 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.813251    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfac97c-292e-4b27-aeb7-acad5b69564f-webhook-cert\") pod \"maas-controller-7955c6964b-742pj\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") " pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.815899 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.815877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfac97c-292e-4b27-aeb7-acad5b69564f-webhook-cert\") pod \"maas-controller-7955c6964b-742pj\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") " pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.831736 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.831710    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k6gm\" (UniqueName: \"kubernetes.io/projected/7cfac97c-292e-4b27-aeb7-acad5b69564f-kube-api-access-4k6gm\") pod \"maas-controller-7955c6964b-742pj\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") " pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:46.876774 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:46.876753    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:47.008405 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:47.008381    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955c6964b-742pj"]
Jul 21 17:10:47.010666 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:10:47.010634    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfac97c_292e_4b27_aeb7_acad5b69564f.slice/crio-c67e794224fafea78d4add8d706b09b6eb46e1edd6f3c767715b6b219f0f83ba WatchSource:0}: Error finding container c67e794224fafea78d4add8d706b09b6eb46e1edd6f3c767715b6b219f0f83ba: Status 404 returned error can't find the container with id c67e794224fafea78d4add8d706b09b6eb46e1edd6f3c767715b6b219f0f83ba
Jul 21 17:10:48.001918 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:48.001802    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955c6964b-742pj" event={"ID":"7cfac97c-292e-4b27-aeb7-acad5b69564f","Type":"ContainerStarted","Data":"c67e794224fafea78d4add8d706b09b6eb46e1edd6f3c767715b6b219f0f83ba"}
Jul 21 17:10:50.014582 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:50.014523    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955c6964b-742pj" event={"ID":"7cfac97c-292e-4b27-aeb7-acad5b69564f","Type":"ContainerStarted","Data":"5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5"}
Jul 21 17:10:50.014968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:50.014730    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:10:50.035368 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:10:50.035315    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7955c6964b-742pj" podStartSLOduration=1.6786802330000001 podStartE2EDuration="4.035297752s" totalImagesPullingTime="2.356617519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:10:50.032581291 +0000 UTC m=+529.997480891" watchObservedRunningTime="2026-07-21 17:10:50.035297752 +0000 UTC m=+530.000197356"
Jul 21 17:11:00.782906 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:00.782820    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7955c6964b-742pj"]
Jul 21 17:11:00.783436 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:00.783074    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7955c6964b-742pj" podUID="7cfac97c-292e-4b27-aeb7-acad5b69564f" containerName="manager" containerID="cri-o://5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5" gracePeriod=30
Jul 21 17:11:00.785030 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:00.785007    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:11:01.030985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.030960    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:11:01.071365 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.071290    2561 generic.go:356] "Generic (PLEG): container finished" podID="7cfac97c-292e-4b27-aeb7-acad5b69564f" containerID="5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5" exitCode=0
Jul 21 17:11:01.071365 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.071338    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955c6964b-742pj"
Jul 21 17:11:01.071535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.071385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955c6964b-742pj" event={"ID":"7cfac97c-292e-4b27-aeb7-acad5b69564f","Type":"ContainerDied","Data":"5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5"}
Jul 21 17:11:01.071535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.071452    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955c6964b-742pj" event={"ID":"7cfac97c-292e-4b27-aeb7-acad5b69564f","Type":"ContainerDied","Data":"c67e794224fafea78d4add8d706b09b6eb46e1edd6f3c767715b6b219f0f83ba"}
Jul 21 17:11:01.071535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.071492    2561 scope.go:117] "RemoveContainer" containerID="5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5"
Jul 21 17:11:01.082062 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.081778    2561 scope.go:117] "RemoveContainer" containerID="5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5"
Jul 21 17:11:01.082297 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:11:01.082272    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5\": container with ID starting with 5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5 not found: ID does not exist" containerID="5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5"
Jul 21 17:11:01.082365 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.082308    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5"} err="failed to get container status \"5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5\": rpc error: code = NotFound desc = could not find container \"5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5\": container with ID starting with 5a993f7b2af543999d70187e4d1d900ff8b6cabd96c9a6e613daba9ba6d8c7c5 not found: ID does not exist"
Jul 21 17:11:01.143621 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.143586    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfac97c-292e-4b27-aeb7-acad5b69564f-webhook-cert\") pod \"7cfac97c-292e-4b27-aeb7-acad5b69564f\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") "
Jul 21 17:11:01.143793 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.143664    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k6gm\" (UniqueName: \"kubernetes.io/projected/7cfac97c-292e-4b27-aeb7-acad5b69564f-kube-api-access-4k6gm\") pod \"7cfac97c-292e-4b27-aeb7-acad5b69564f\" (UID: \"7cfac97c-292e-4b27-aeb7-acad5b69564f\") "
Jul 21 17:11:01.145950 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.145921    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfac97c-292e-4b27-aeb7-acad5b69564f-kube-api-access-4k6gm" (OuterVolumeSpecName: "kube-api-access-4k6gm") pod "7cfac97c-292e-4b27-aeb7-acad5b69564f" (UID: "7cfac97c-292e-4b27-aeb7-acad5b69564f"). InnerVolumeSpecName "kube-api-access-4k6gm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:11:01.145950 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.145940    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cfac97c-292e-4b27-aeb7-acad5b69564f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7cfac97c-292e-4b27-aeb7-acad5b69564f" (UID: "7cfac97c-292e-4b27-aeb7-acad5b69564f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:11:01.244363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.244318    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cfac97c-292e-4b27-aeb7-acad5b69564f-webhook-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:11:01.244363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.244359    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k6gm\" (UniqueName: \"kubernetes.io/projected/7cfac97c-292e-4b27-aeb7-acad5b69564f-kube-api-access-4k6gm\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:11:01.395309 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.395270    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7955c6964b-742pj"]
Jul 21 17:11:01.400051 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:01.400020    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7955c6964b-742pj"]
Jul 21 17:11:02.513280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:02.513247    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cfac97c-292e-4b27-aeb7-acad5b69564f" path="/var/lib/kubelet/pods/7cfac97c-292e-4b27-aeb7-acad5b69564f/volumes"
Jul 21 17:11:22.103911 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.103872    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-m2q87"]
Jul 21 17:11:22.104491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.104364    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cfac97c-292e-4b27-aeb7-acad5b69564f" containerName="manager"
Jul 21 17:11:22.104491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.104382    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cfac97c-292e-4b27-aeb7-acad5b69564f" containerName="manager"
Jul 21 17:11:22.104491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.104467    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cfac97c-292e-4b27-aeb7-acad5b69564f" containerName="manager"
Jul 21 17:11:22.112074 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.112046    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.114615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.114590    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xct8l\""
Jul 21 17:11:22.115422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.115400    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 17:11:22.120617 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.120595    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-m2q87"]
Jul 21 17:11:22.215325 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.215292    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fe5898-8eb0-49fe-9b23-1e14a01e253c-plugins-config-volume\") pod \"payload-processing-5bf47cd988-m2q87\" (UID: \"15fe5898-8eb0-49fe-9b23-1e14a01e253c\") " pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.215471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.215362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwccd\" (UniqueName: \"kubernetes.io/projected/15fe5898-8eb0-49fe-9b23-1e14a01e253c-kube-api-access-pwccd\") pod \"payload-processing-5bf47cd988-m2q87\" (UID: \"15fe5898-8eb0-49fe-9b23-1e14a01e253c\") " pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.316608 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.316542    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fe5898-8eb0-49fe-9b23-1e14a01e253c-plugins-config-volume\") pod \"payload-processing-5bf47cd988-m2q87\" (UID: \"15fe5898-8eb0-49fe-9b23-1e14a01e253c\") " pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.316768 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.316641    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwccd\" (UniqueName: \"kubernetes.io/projected/15fe5898-8eb0-49fe-9b23-1e14a01e253c-kube-api-access-pwccd\") pod \"payload-processing-5bf47cd988-m2q87\" (UID: \"15fe5898-8eb0-49fe-9b23-1e14a01e253c\") " pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.317274 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.317256    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15fe5898-8eb0-49fe-9b23-1e14a01e253c-plugins-config-volume\") pod \"payload-processing-5bf47cd988-m2q87\" (UID: \"15fe5898-8eb0-49fe-9b23-1e14a01e253c\") " pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.327900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.327877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwccd\" (UniqueName: \"kubernetes.io/projected/15fe5898-8eb0-49fe-9b23-1e14a01e253c-kube-api-access-pwccd\") pod \"payload-processing-5bf47cd988-m2q87\" (UID: \"15fe5898-8eb0-49fe-9b23-1e14a01e253c\") " pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.423986 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.423960    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:22.581651 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:22.569170    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-m2q87"]
Jul 21 17:11:22.581651 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:11:22.572729    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fe5898_8eb0_49fe_9b23_1e14a01e253c.slice/crio-8cdf58d82a288d3546a1fbaccb1e021fdfd7d28b8590c697b5e22ef49666d973 WatchSource:0}: Error finding container 8cdf58d82a288d3546a1fbaccb1e021fdfd7d28b8590c697b5e22ef49666d973: Status 404 returned error can't find the container with id 8cdf58d82a288d3546a1fbaccb1e021fdfd7d28b8590c697b5e22ef49666d973
Jul 21 17:11:23.191061 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:23.191017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87" event={"ID":"15fe5898-8eb0-49fe-9b23-1e14a01e253c","Type":"ContainerStarted","Data":"8cdf58d82a288d3546a1fbaccb1e021fdfd7d28b8590c697b5e22ef49666d973"}
Jul 21 17:11:24.197605 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:24.197570    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87" event={"ID":"15fe5898-8eb0-49fe-9b23-1e14a01e253c","Type":"ContainerStarted","Data":"3b8225b6ef039d2709a5cf2a335bcbfb5d9bc45adef6bd36383280bc18f63d83"}
Jul 21 17:11:24.197905 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:24.197676    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:24.216054 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:24.216010    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87" podStartSLOduration=0.665494241 podStartE2EDuration="2.215994798s" totalImagesPullingTime="1.550500557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:11:24.214433608 +0000 UTC m=+564.179333213" watchObservedRunningTime="2026-07-21 17:11:24.215994798 +0000 UTC m=+564.180894404"
Jul 21 17:11:35.205146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:35.205115    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-m2q87"
Jul 21 17:11:57.320002 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.319960    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"]
Jul 21 17:11:57.324989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.324966    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.327407 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.327386    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 17:11:57.327519 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.327406    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 17:11:57.328612 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.328588    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k6wdl\""
Jul 21 17:11:57.328832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.328810    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 21 17:11:57.342843 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.341549    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"]
Jul 21 17:11:57.402418 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.402382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.402418 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.402425    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.402664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.402472    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g77gn\" (UniqueName: \"kubernetes.io/projected/f520906f-c8e5-4cbb-adb6-a80ce36f1431-kube-api-access-g77gn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.402664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.402491    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.402664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.402511    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.402664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.402532    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f520906f-c8e5-4cbb-adb6-a80ce36f1431-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503203 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503203 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f520906f-c8e5-4cbb-adb6-a80ce36f1431-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503405 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503443 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503435    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503512 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503486    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g77gn\" (UniqueName: \"kubernetes.io/projected/f520906f-c8e5-4cbb-adb6-a80ce36f1431-kube-api-access-g77gn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503589 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503685 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503653    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.503827 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503812    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.504024 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.503999    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.505817 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.505798    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f520906f-c8e5-4cbb-adb6-a80ce36f1431-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.505888 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.505820    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f520906f-c8e5-4cbb-adb6-a80ce36f1431-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.512471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.512443    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g77gn\" (UniqueName: \"kubernetes.io/projected/f520906f-c8e5-4cbb-adb6-a80ce36f1431-kube-api-access-g77gn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n4dqc\" (UID: \"f520906f-c8e5-4cbb-adb6-a80ce36f1431\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.644215 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.644183    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:11:57.789586 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:57.789544    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"]
Jul 21 17:11:57.791981 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:11:57.791959    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf520906f_c8e5_4cbb_adb6_a80ce36f1431.slice/crio-add3adbffcb65e3952657241e5c065f61fecfce7b06951f6a5416b1cba6f7d6f WatchSource:0}: Error finding container add3adbffcb65e3952657241e5c065f61fecfce7b06951f6a5416b1cba6f7d6f: Status 404 returned error can't find the container with id add3adbffcb65e3952657241e5c065f61fecfce7b06951f6a5416b1cba6f7d6f
Jul 21 17:11:58.377627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:58.377550    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc" event={"ID":"f520906f-c8e5-4cbb-adb6-a80ce36f1431","Type":"ContainerStarted","Data":"add3adbffcb65e3952657241e5c065f61fecfce7b06951f6a5416b1cba6f7d6f"}
Jul 21 17:11:59.383840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:59.383754    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc" event={"ID":"f520906f-c8e5-4cbb-adb6-a80ce36f1431","Type":"ContainerStarted","Data":"085a725d6922bb4457a2888353ef5aa1d10e2b8805fe6cd9ebc200c1fd163d4f"}
Jul 21 17:11:59.403989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:11:59.403947    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc" podStartSLOduration=1.255971863 podStartE2EDuration="2.403930862s" totalImagesPullingTime="1.147958999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:11:59.402881721 +0000 UTC m=+599.367781325" watchObservedRunningTime="2026-07-21 17:11:59.403930862 +0000 UTC m=+599.368830466"
Jul 21 17:12:00.463080 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:00.463049    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:12:00.464686 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:00.464668    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:12:07.644460 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:07.644410    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:12:07.644985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:07.644484    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:12:07.657413 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:07.657388    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:12:08.438952 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:08.438914    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n4dqc"
Jul 21 17:12:26.531219 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.531133    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"]
Jul 21 17:12:26.534595 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.534570    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.536923 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.536904    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 17:12:26.547263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.547237    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"]
Jul 21 17:12:26.643155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.643123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.643310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.643162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v97w\" (UniqueName: \"kubernetes.io/projected/2b4321cc-29f2-4bce-99be-1cd709c78cb9-kube-api-access-8v97w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.643310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.643197    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.643310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.643247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.643310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.643268    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b4321cc-29f2-4bce-99be-1cd709c78cb9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.643310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.643285    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744319 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744287    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744350    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b4321cc-29f2-4bce-99be-1cd709c78cb9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744376    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744445    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744469    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v97w\" (UniqueName: \"kubernetes.io/projected/2b4321cc-29f2-4bce-99be-1cd709c78cb9-kube-api-access-8v97w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744779    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744834 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.744900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.744877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.746895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.746875    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b4321cc-29f2-4bce-99be-1cd709c78cb9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.747029 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.747010    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b4321cc-29f2-4bce-99be-1cd709c78cb9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.757464 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.757436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v97w\" (UniqueName: \"kubernetes.io/projected/2b4321cc-29f2-4bce-99be-1cd709c78cb9-kube-api-access-8v97w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zqbcg\" (UID: \"2b4321cc-29f2-4bce-99be-1cd709c78cb9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.845440 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.845380    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:26.980240 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.980212    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"]
Jul 21 17:12:26.983122 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:12:26.983097    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4321cc_29f2_4bce_99be_1cd709c78cb9.slice/crio-62c48680a9896f5f6e8ed712876f65653841bd55d1e7ca627bb38612bc9558d8 WatchSource:0}: Error finding container 62c48680a9896f5f6e8ed712876f65653841bd55d1e7ca627bb38612bc9558d8: Status 404 returned error can't find the container with id 62c48680a9896f5f6e8ed712876f65653841bd55d1e7ca627bb38612bc9558d8
Jul 21 17:12:26.984392 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:26.984377    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:12:27.531606 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:27.531500    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg" event={"ID":"2b4321cc-29f2-4bce-99be-1cd709c78cb9","Type":"ContainerStarted","Data":"82163377ac1c957fd81e287a9a15bc16b3b0b4d12a64eae2c570d3a655808457"}
Jul 21 17:12:27.531606 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:27.531590    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg" event={"ID":"2b4321cc-29f2-4bce-99be-1cd709c78cb9","Type":"ContainerStarted","Data":"62c48680a9896f5f6e8ed712876f65653841bd55d1e7ca627bb38612bc9558d8"}
Jul 21 17:12:27.552038 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:27.551988    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg" podStartSLOduration=1.355104093 podStartE2EDuration="1.551972496s" totalImagesPullingTime="196.868403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:12:27.550292183 +0000 UTC m=+627.515191793" watchObservedRunningTime="2026-07-21 17:12:27.551972496 +0000 UTC m=+627.516872100"
Jul 21 17:12:36.846153 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:36.846097    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:36.846796 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:36.846189    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:36.863360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:36.863331    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:12:37.596715 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:12:37.596688    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zqbcg"
Jul 21 17:13:49.457223 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.457182    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6494d5c85b-45klz"]
Jul 21 17:13:49.457723 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.457404    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6494d5c85b-45klz" podUID="e36f22cf-a567-4d65-8630-ab214debcecf" containerName="authorino" containerID="cri-o://802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc" gracePeriod=30
Jul 21 17:13:49.706849 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.706818    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:13:49.863840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.863765    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6trtv\" (UniqueName: \"kubernetes.io/projected/e36f22cf-a567-4d65-8630-ab214debcecf-kube-api-access-6trtv\") pod \"e36f22cf-a567-4d65-8630-ab214debcecf\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") "
Jul 21 17:13:49.863840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.863818    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e36f22cf-a567-4d65-8630-ab214debcecf-tls-cert\") pod \"e36f22cf-a567-4d65-8630-ab214debcecf\" (UID: \"e36f22cf-a567-4d65-8630-ab214debcecf\") "
Jul 21 17:13:49.865971 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.865936    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36f22cf-a567-4d65-8630-ab214debcecf-kube-api-access-6trtv" (OuterVolumeSpecName: "kube-api-access-6trtv") pod "e36f22cf-a567-4d65-8630-ab214debcecf" (UID: "e36f22cf-a567-4d65-8630-ab214debcecf"). InnerVolumeSpecName "kube-api-access-6trtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:13:49.873670 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.873642    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36f22cf-a567-4d65-8630-ab214debcecf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e36f22cf-a567-4d65-8630-ab214debcecf" (UID: "e36f22cf-a567-4d65-8630-ab214debcecf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:13:49.965245 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.965220    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6trtv\" (UniqueName: \"kubernetes.io/projected/e36f22cf-a567-4d65-8630-ab214debcecf-kube-api-access-6trtv\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:13:49.965245 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.965244    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e36f22cf-a567-4d65-8630-ab214debcecf-tls-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:13:49.967352 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.967325    2561 generic.go:356] "Generic (PLEG): container finished" podID="e36f22cf-a567-4d65-8630-ab214debcecf" containerID="802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc" exitCode=0
Jul 21 17:13:49.967482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.967364    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6494d5c85b-45klz" event={"ID":"e36f22cf-a567-4d65-8630-ab214debcecf","Type":"ContainerDied","Data":"802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc"}
Jul 21 17:13:49.967482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.967372    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6494d5c85b-45klz"
Jul 21 17:13:49.967482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.967398    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6494d5c85b-45klz" event={"ID":"e36f22cf-a567-4d65-8630-ab214debcecf","Type":"ContainerDied","Data":"68ab535383c52d06b70bbbaf1d9f051f15e05572fd752c5b8b3b63cb196e6c1c"}
Jul 21 17:13:49.967482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.967419    2561 scope.go:117] "RemoveContainer" containerID="802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc"
Jul 21 17:13:49.977390 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.977373    2561 scope.go:117] "RemoveContainer" containerID="802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc"
Jul 21 17:13:49.977652 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:13:49.977630    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc\": container with ID starting with 802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc not found: ID does not exist" containerID="802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc"
Jul 21 17:13:49.977713 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:49.977661    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc"} err="failed to get container status \"802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc\": rpc error: code = NotFound desc = could not find container \"802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc\": container with ID starting with 802887a6129b76b8d49b25478aba147e2b4f04b5e63a38c9df9af6bccd2bb7bc not found: ID does not exist"
Jul 21 17:13:50.005054 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:50.005021    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6494d5c85b-45klz"]
Jul 21 17:13:50.010363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:50.010340    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6494d5c85b-45klz"]
Jul 21 17:13:50.512919 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:13:50.512888    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36f22cf-a567-4d65-8630-ab214debcecf" path="/var/lib/kubelet/pods/e36f22cf-a567-4d65-8630-ab214debcecf/volumes"
Jul 21 17:15:36.660185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.660104    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68d5cf45d-p7bxs"]
Jul 21 17:15:36.660672 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.660455    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36f22cf-a567-4d65-8630-ab214debcecf" containerName="authorino"
Jul 21 17:15:36.660672 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.660465    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36f22cf-a567-4d65-8630-ab214debcecf" containerName="authorino"
Jul 21 17:15:36.660672 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.660520    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36f22cf-a567-4d65-8630-ab214debcecf" containerName="authorino"
Jul 21 17:15:36.669136 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.669110    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.672748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.672375    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rk7d7\""
Jul 21 17:15:36.672748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.672462    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 17:15:36.672748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.672656    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 17:15:36.674848 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.674818    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d5cf45d-p7bxs"]
Jul 21 17:15:36.757373 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.757340    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6hsv\" (UniqueName: \"kubernetes.io/projected/d510fa59-82af-4b26-b4d4-10d481ff613c-kube-api-access-k6hsv\") pod \"maas-controller-68d5cf45d-p7bxs\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") " pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.757587 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.757382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d510fa59-82af-4b26-b4d4-10d481ff613c-webhook-cert\") pod \"maas-controller-68d5cf45d-p7bxs\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") " pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.857823 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.857782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d510fa59-82af-4b26-b4d4-10d481ff613c-webhook-cert\") pod \"maas-controller-68d5cf45d-p7bxs\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") " pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.858022 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.857915    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6hsv\" (UniqueName: \"kubernetes.io/projected/d510fa59-82af-4b26-b4d4-10d481ff613c-kube-api-access-k6hsv\") pod \"maas-controller-68d5cf45d-p7bxs\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") " pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.860530 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.860500    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d510fa59-82af-4b26-b4d4-10d481ff613c-webhook-cert\") pod \"maas-controller-68d5cf45d-p7bxs\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") " pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.868364 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.868333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6hsv\" (UniqueName: \"kubernetes.io/projected/d510fa59-82af-4b26-b4d4-10d481ff613c-kube-api-access-k6hsv\") pod \"maas-controller-68d5cf45d-p7bxs\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") " pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:36.980910 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:36.980821    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:37.119678 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:37.119655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68d5cf45d-p7bxs"]
Jul 21 17:15:37.122060 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:15:37.122034    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd510fa59_82af_4b26_b4d4_10d481ff613c.slice/crio-fc67a2d7de5f3baa30670c828cd4343e8d78ddaf188b94d3f659a4c3ef0e6e09 WatchSource:0}: Error finding container fc67a2d7de5f3baa30670c828cd4343e8d78ddaf188b94d3f659a4c3ef0e6e09: Status 404 returned error can't find the container with id fc67a2d7de5f3baa30670c828cd4343e8d78ddaf188b94d3f659a4c3ef0e6e09
Jul 21 17:15:37.518311 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:37.518276    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d5cf45d-p7bxs" event={"ID":"d510fa59-82af-4b26-b4d4-10d481ff613c","Type":"ContainerStarted","Data":"fc67a2d7de5f3baa30670c828cd4343e8d78ddaf188b94d3f659a4c3ef0e6e09"}
Jul 21 17:15:38.524195 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:38.524157    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d5cf45d-p7bxs" event={"ID":"d510fa59-82af-4b26-b4d4-10d481ff613c","Type":"ContainerStarted","Data":"58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771"}
Jul 21 17:15:38.524613 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:38.524245    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:38.546843 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:38.546794    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68d5cf45d-p7bxs" podStartSLOduration=2.100039219 podStartE2EDuration="2.546778565s" totalImagesPullingTime="446.739346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:15:38.545430198 +0000 UTC m=+818.510329804" watchObservedRunningTime="2026-07-21 17:15:38.546778565 +0000 UTC m=+818.511678224"
Jul 21 17:15:49.530998 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:49.530970    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:54.159686 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.159646    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68d5cf45d-p7bxs"]
Jul 21 17:15:54.160070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.159923    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68d5cf45d-p7bxs" podUID="d510fa59-82af-4b26-b4d4-10d481ff613c" containerName="manager" containerID="cri-o://58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771" gracePeriod=30
Jul 21 17:15:54.407727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.407695    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:54.505714 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.505634    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d510fa59-82af-4b26-b4d4-10d481ff613c-webhook-cert\") pod \"d510fa59-82af-4b26-b4d4-10d481ff613c\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") "
Jul 21 17:15:54.505875 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.505741    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6hsv\" (UniqueName: \"kubernetes.io/projected/d510fa59-82af-4b26-b4d4-10d481ff613c-kube-api-access-k6hsv\") pod \"d510fa59-82af-4b26-b4d4-10d481ff613c\" (UID: \"d510fa59-82af-4b26-b4d4-10d481ff613c\") "
Jul 21 17:15:54.507816 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.507778    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d510fa59-82af-4b26-b4d4-10d481ff613c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d510fa59-82af-4b26-b4d4-10d481ff613c" (UID: "d510fa59-82af-4b26-b4d4-10d481ff613c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:15:54.507931 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.507864    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d510fa59-82af-4b26-b4d4-10d481ff613c-kube-api-access-k6hsv" (OuterVolumeSpecName: "kube-api-access-k6hsv") pod "d510fa59-82af-4b26-b4d4-10d481ff613c" (UID: "d510fa59-82af-4b26-b4d4-10d481ff613c"). InnerVolumeSpecName "kube-api-access-k6hsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:15:54.603295 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.603263    2561 generic.go:356] "Generic (PLEG): container finished" podID="d510fa59-82af-4b26-b4d4-10d481ff613c" containerID="58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771" exitCode=0
Jul 21 17:15:54.603456 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.603309    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68d5cf45d-p7bxs"
Jul 21 17:15:54.603456 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.603348    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d5cf45d-p7bxs" event={"ID":"d510fa59-82af-4b26-b4d4-10d481ff613c","Type":"ContainerDied","Data":"58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771"}
Jul 21 17:15:54.603456 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.603399    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68d5cf45d-p7bxs" event={"ID":"d510fa59-82af-4b26-b4d4-10d481ff613c","Type":"ContainerDied","Data":"fc67a2d7de5f3baa30670c828cd4343e8d78ddaf188b94d3f659a4c3ef0e6e09"}
Jul 21 17:15:54.603456 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.603428    2561 scope.go:117] "RemoveContainer" containerID="58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771"
Jul 21 17:15:54.607138 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.607116    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6hsv\" (UniqueName: \"kubernetes.io/projected/d510fa59-82af-4b26-b4d4-10d481ff613c-kube-api-access-k6hsv\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:15:54.607138 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.607140    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d510fa59-82af-4b26-b4d4-10d481ff613c-webhook-cert\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:15:54.612309 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.612290    2561 scope.go:117] "RemoveContainer" containerID="58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771"
Jul 21 17:15:54.612619 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:15:54.612549    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771\": container with ID starting with 58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771 not found: ID does not exist" containerID="58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771"
Jul 21 17:15:54.612619 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.612591    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771"} err="failed to get container status \"58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771\": rpc error: code = NotFound desc = could not find container \"58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771\": container with ID starting with 58aa8576e4da6573622c235c26694ca6fdfeef3ab96902eaf883795333efd771 not found: ID does not exist"
Jul 21 17:15:54.622840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.622816    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68d5cf45d-p7bxs"]
Jul 21 17:15:54.627701 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:54.627679    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68d5cf45d-p7bxs"]
Jul 21 17:15:56.513023 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:15:56.512991    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d510fa59-82af-4b26-b4d4-10d481ff613c" path="/var/lib/kubelet/pods/d510fa59-82af-4b26-b4d4-10d481ff613c/volumes"
Jul 21 17:17:00.494387 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:17:00.494318    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:17:00.496709 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:17:00.496688    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:20:36.839312 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:36.839276    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"]
Jul 21 17:20:37.060201 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:37.060161    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld" podUID="c76014de-0db5-4f2f-a3ba-bd3eaadc1742" containerName="manager" containerID="cri-o://5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa" gracePeriod=10
Jul 21 17:20:39.711932 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.711905    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:20:39.842370 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.842278    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-extensions-socket-volume\") pod \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") "
Jul 21 17:20:39.842370 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.842324    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb85q\" (UniqueName: \"kubernetes.io/projected/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-kube-api-access-tb85q\") pod \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\" (UID: \"c76014de-0db5-4f2f-a3ba-bd3eaadc1742\") "
Jul 21 17:20:39.842705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.842682    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c76014de-0db5-4f2f-a3ba-bd3eaadc1742" (UID: "c76014de-0db5-4f2f-a3ba-bd3eaadc1742"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:20:39.844502 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.844473    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-kube-api-access-tb85q" (OuterVolumeSpecName: "kube-api-access-tb85q") pod "c76014de-0db5-4f2f-a3ba-bd3eaadc1742" (UID: "c76014de-0db5-4f2f-a3ba-bd3eaadc1742"). InnerVolumeSpecName "kube-api-access-tb85q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:20:39.943985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.943945    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-extensions-socket-volume\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:20:39.943985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:39.943977    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb85q\" (UniqueName: \"kubernetes.io/projected/c76014de-0db5-4f2f-a3ba-bd3eaadc1742-kube-api-access-tb85q\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:20:40.077136 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.077102    2561 generic.go:356] "Generic (PLEG): container finished" podID="c76014de-0db5-4f2f-a3ba-bd3eaadc1742" containerID="5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa" exitCode=0
Jul 21 17:20:40.077316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.077144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld" event={"ID":"c76014de-0db5-4f2f-a3ba-bd3eaadc1742","Type":"ContainerDied","Data":"5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa"}
Jul 21 17:20:40.077316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.077160    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"
Jul 21 17:20:40.077316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.077220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld" event={"ID":"c76014de-0db5-4f2f-a3ba-bd3eaadc1742","Type":"ContainerDied","Data":"78bba8452f320e818e255eb1c3d384c2b83a7c0936c05e9369e93a8abe85a245"}
Jul 21 17:20:40.077316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.077245    2561 scope.go:117] "RemoveContainer" containerID="5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa"
Jul 21 17:20:40.087777 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.087752    2561 scope.go:117] "RemoveContainer" containerID="5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa"
Jul 21 17:20:40.088023 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:20:40.088007    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa\": container with ID starting with 5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa not found: ID does not exist" containerID="5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa"
Jul 21 17:20:40.088075 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.088030    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa"} err="failed to get container status \"5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa\": rpc error: code = NotFound desc = could not find container \"5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa\": container with ID starting with 5b7900b1c332f96382150b9369a559e0bc1971cedb60491f905da3b94dbfd8aa not found: ID does not exist"
Jul 21 17:20:40.105835 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.105754    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"]
Jul 21 17:20:40.110545 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.110523    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsfld"]
Jul 21 17:20:40.513279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:20:40.513245    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c76014de-0db5-4f2f-a3ba-bd3eaadc1742" path="/var/lib/kubelet/pods/c76014de-0db5-4f2f-a3ba-bd3eaadc1742/volumes"
Jul 21 17:21:38.009859 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.009778    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"]
Jul 21 17:21:38.010320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.010143    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d510fa59-82af-4b26-b4d4-10d481ff613c" containerName="manager"
Jul 21 17:21:38.010320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.010155    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d510fa59-82af-4b26-b4d4-10d481ff613c" containerName="manager"
Jul 21 17:21:38.010320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.010177    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76014de-0db5-4f2f-a3ba-bd3eaadc1742" containerName="manager"
Jul 21 17:21:38.010320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.010183    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76014de-0db5-4f2f-a3ba-bd3eaadc1742" containerName="manager"
Jul 21 17:21:38.010320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.010233    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d510fa59-82af-4b26-b4d4-10d481ff613c" containerName="manager"
Jul 21 17:21:38.010320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.010243    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76014de-0db5-4f2f-a3ba-bd3eaadc1742" containerName="manager"
Jul 21 17:21:38.013257 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.013241    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.016513 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.016494    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bc4hz\""
Jul 21 17:21:38.026763 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.026741    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"]
Jul 21 17:21:38.128160 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.128127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07e45df4-78c2-4c31-9da0-e9abece95f6b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6zptj\" (UID: \"07e45df4-78c2-4c31-9da0-e9abece95f6b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.128306 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.128185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvlpz\" (UniqueName: \"kubernetes.io/projected/07e45df4-78c2-4c31-9da0-e9abece95f6b-kube-api-access-wvlpz\") pod \"kuadrant-operator-controller-manager-86878c769f-6zptj\" (UID: \"07e45df4-78c2-4c31-9da0-e9abece95f6b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.228649 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.228619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvlpz\" (UniqueName: \"kubernetes.io/projected/07e45df4-78c2-4c31-9da0-e9abece95f6b-kube-api-access-wvlpz\") pod \"kuadrant-operator-controller-manager-86878c769f-6zptj\" (UID: \"07e45df4-78c2-4c31-9da0-e9abece95f6b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.228797 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.228713    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07e45df4-78c2-4c31-9da0-e9abece95f6b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6zptj\" (UID: \"07e45df4-78c2-4c31-9da0-e9abece95f6b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.229070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.229050    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07e45df4-78c2-4c31-9da0-e9abece95f6b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6zptj\" (UID: \"07e45df4-78c2-4c31-9da0-e9abece95f6b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.238673 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.238651    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvlpz\" (UniqueName: \"kubernetes.io/projected/07e45df4-78c2-4c31-9da0-e9abece95f6b-kube-api-access-wvlpz\") pod \"kuadrant-operator-controller-manager-86878c769f-6zptj\" (UID: \"07e45df4-78c2-4c31-9da0-e9abece95f6b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.323230 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.323163    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:38.464204 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:38.464180    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"]
Jul 21 17:21:38.466291 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:21:38.466265    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e45df4_78c2_4c31_9da0_e9abece95f6b.slice/crio-a3341f086905981b5cd805285a082ea93f977e8cf53c9f739ba44e1e8eda1f47 WatchSource:0}: Error finding container a3341f086905981b5cd805285a082ea93f977e8cf53c9f739ba44e1e8eda1f47: Status 404 returned error can't find the container with id a3341f086905981b5cd805285a082ea93f977e8cf53c9f739ba44e1e8eda1f47
Jul 21 17:21:39.387964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:39.387919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj" event={"ID":"07e45df4-78c2-4c31-9da0-e9abece95f6b","Type":"ContainerStarted","Data":"aefc35cb805e9df94b2cb2b6c7c0714289ce1ee5e1bc031c71868db45b698d06"}
Jul 21 17:21:39.388460 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:39.388016    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj" event={"ID":"07e45df4-78c2-4c31-9da0-e9abece95f6b","Type":"ContainerStarted","Data":"a3341f086905981b5cd805285a082ea93f977e8cf53c9f739ba44e1e8eda1f47"}
Jul 21 17:21:39.388460 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:39.388048    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:21:39.412992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:39.412935    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj" podStartSLOduration=2.412917061 podStartE2EDuration="2.412917061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:21:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:21:39.41086542 +0000 UTC m=+1179.375765023" watchObservedRunningTime="2026-07-21 17:21:39.412917061 +0000 UTC m=+1179.377816667"
Jul 21 17:21:50.394652 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:21:50.394623    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zptj"
Jul 21 17:22:00.527529 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:22:00.527497    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:22:00.530824 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:22:00.530801    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:24:24.643494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.643415    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"]
Jul 21 17:24:24.646923 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.646903    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.649677 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.649648    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-003af18a-kserve-self-signed-certs\""
Jul 21 17:24:24.670747 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.670716    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"]
Jul 21 17:24:24.782394 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.782362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-dshm\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.782552 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.782412    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-model-cache\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.782552 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.782441    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tls-certs\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.782552 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.782484    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbtrg\" (UniqueName: \"kubernetes.io/projected/f0df793c-0c7f-468f-b319-1c9c1e1ad934-kube-api-access-nbtrg\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.782713 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.782546    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tmp-dir\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.782713 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.782608    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-home\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.883654 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.883619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-model-cache\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.883654 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.883655    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tls-certs\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.883863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.883725    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbtrg\" (UniqueName: \"kubernetes.io/projected/f0df793c-0c7f-468f-b319-1c9c1e1ad934-kube-api-access-nbtrg\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.883863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.883776    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tmp-dir\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.883863 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.883813    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-home\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.884022 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.883875    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-dshm\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.884071 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.884047    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-model-cache\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.884129 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.884073    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tmp-dir\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.884248 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.884198    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-home\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.886110 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.886091    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-dshm\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.886377 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.886360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tls-certs\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.893079 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.893053    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbtrg\" (UniqueName: \"kubernetes.io/projected/f0df793c-0c7f-468f-b319-1c9c1e1ad934-kube-api-access-nbtrg\") pod \"e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") " pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:24.958254 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:24.958186    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:25.096266 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:25.096225    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"]
Jul 21 17:24:25.103457 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:24:25.103431    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0df793c_0c7f_468f_b319_1c9c1e1ad934.slice/crio-159896b0a697130eef279ca85788866b8921068d836dc44828a103d5e2a10cc2 WatchSource:0}: Error finding container 159896b0a697130eef279ca85788866b8921068d836dc44828a103d5e2a10cc2: Status 404 returned error can't find the container with id 159896b0a697130eef279ca85788866b8921068d836dc44828a103d5e2a10cc2
Jul 21 17:24:25.258902 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:25.258813    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw" event={"ID":"f0df793c-0c7f-468f-b319-1c9c1e1ad934","Type":"ContainerStarted","Data":"1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12"}
Jul 21 17:24:25.258902 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:25.258865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw" event={"ID":"f0df793c-0c7f-468f-b319-1c9c1e1ad934","Type":"ContainerStarted","Data":"159896b0a697130eef279ca85788866b8921068d836dc44828a103d5e2a10cc2"}
Jul 21 17:24:25.282978 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:25.282928    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw" podStartSLOduration=1.282912286 podStartE2EDuration="1.282912286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:24:25.280654061 +0000 UTC m=+1345.245553666" watchObservedRunningTime="2026-07-21 17:24:25.282912286 +0000 UTC m=+1345.247811889"
Jul 21 17:24:34.959242 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:34.959201    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:34.959838 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:34.959263    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:34.972689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:34.972657    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:24:35.321741 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:24:35.321661    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:25:05.974902 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:05.974867    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"]
Jul 21 17:25:20.995220 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:20.995179    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw" podUID="f0df793c-0c7f-468f-b319-1c9c1e1ad934" containerName="main" containerID="cri-o://1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12" gracePeriod=45
Jul 21 17:25:21.246959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.246902    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:25:21.270051 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270023    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-model-cache\") pod \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") "
Jul 21 17:25:21.270188 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270065    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-dshm\") pod \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") "
Jul 21 17:25:21.270188 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270161    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tls-certs\") pod \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") "
Jul 21 17:25:21.270316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270217    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbtrg\" (UniqueName: \"kubernetes.io/projected/f0df793c-0c7f-468f-b319-1c9c1e1ad934-kube-api-access-nbtrg\") pod \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") "
Jul 21 17:25:21.270316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270248    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tmp-dir\") pod \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") "
Jul 21 17:25:21.270316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270271    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-home\") pod \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\" (UID: \"f0df793c-0c7f-468f-b319-1c9c1e1ad934\") "
Jul 21 17:25:21.270794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270316    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0df793c-0c7f-468f-b319-1c9c1e1ad934" (UID: "f0df793c-0c7f-468f-b319-1c9c1e1ad934"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:25:21.270794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270527    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-model-cache\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:25:21.270794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270662    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0df793c-0c7f-468f-b319-1c9c1e1ad934" (UID: "f0df793c-0c7f-468f-b319-1c9c1e1ad934"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:25:21.270794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.270747    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-home" (OuterVolumeSpecName: "home") pod "f0df793c-0c7f-468f-b319-1c9c1e1ad934" (UID: "f0df793c-0c7f-468f-b319-1c9c1e1ad934"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:25:21.272890 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.272865    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-dshm" (OuterVolumeSpecName: "dshm") pod "f0df793c-0c7f-468f-b319-1c9c1e1ad934" (UID: "f0df793c-0c7f-468f-b319-1c9c1e1ad934"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:25:21.272996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.272977    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0df793c-0c7f-468f-b319-1c9c1e1ad934-kube-api-access-nbtrg" (OuterVolumeSpecName: "kube-api-access-nbtrg") pod "f0df793c-0c7f-468f-b319-1c9c1e1ad934" (UID: "f0df793c-0c7f-468f-b319-1c9c1e1ad934"). InnerVolumeSpecName "kube-api-access-nbtrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:25:21.273040 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.272985    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0df793c-0c7f-468f-b319-1c9c1e1ad934" (UID: "f0df793c-0c7f-468f-b319-1c9c1e1ad934"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:25:21.371879 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.371848    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tls-certs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:25:21.371879 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.371875    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbtrg\" (UniqueName: \"kubernetes.io/projected/f0df793c-0c7f-468f-b319-1c9c1e1ad934-kube-api-access-nbtrg\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:25:21.371879 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.371884    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-tmp-dir\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:25:21.372074 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.371892    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-home\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:25:21.372074 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.371900    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0df793c-0c7f-468f-b319-1c9c1e1ad934-dshm\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:25:21.552803 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.552714    2561 generic.go:356] "Generic (PLEG): container finished" podID="f0df793c-0c7f-468f-b319-1c9c1e1ad934" containerID="1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12" exitCode=0
Jul 21 17:25:21.552803 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.552761    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"
Jul 21 17:25:21.553024 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.552797    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw" event={"ID":"f0df793c-0c7f-468f-b319-1c9c1e1ad934","Type":"ContainerDied","Data":"1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12"}
Jul 21 17:25:21.553024 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.552843    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw" event={"ID":"f0df793c-0c7f-468f-b319-1c9c1e1ad934","Type":"ContainerDied","Data":"159896b0a697130eef279ca85788866b8921068d836dc44828a103d5e2a10cc2"}
Jul 21 17:25:21.553024 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.552861    2561 scope.go:117] "RemoveContainer" containerID="1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12"
Jul 21 17:25:21.562622 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.562607    2561 scope.go:117] "RemoveContainer" containerID="1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12"
Jul 21 17:25:21.562873 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:25:21.562856    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12\": container with ID starting with 1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12 not found: ID does not exist" containerID="1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12"
Jul 21 17:25:21.562926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.562882    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12"} err="failed to get container status \"1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12\": rpc error: code = NotFound desc = could not find container \"1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12\": container with ID starting with 1706a69cffa29c622e81d3a28be6765133d56fd8652bd1193d202a33a9b61e12 not found: ID does not exist"
Jul 21 17:25:21.575617 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.575593    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"]
Jul 21 17:25:21.580277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:21.580256    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-003af18a-kserve-74c4f77dc9-qfljw"]
Jul 21 17:25:22.513404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:25:22.513367    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0df793c-0c7f-468f-b319-1c9c1e1ad934" path="/var/lib/kubelet/pods/f0df793c-0c7f-468f-b319-1c9c1e1ad934/volumes"
Jul 21 17:27:00.562166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:27:00.562135    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:27:00.565978 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:27:00.565957    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:29:12.982491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.982456    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"]
Jul 21 17:29:12.984792 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.982903    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0df793c-0c7f-468f-b319-1c9c1e1ad934" containerName="main"
Jul 21 17:29:12.984792 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.982919    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0df793c-0c7f-468f-b319-1c9c1e1ad934" containerName="main"
Jul 21 17:29:12.984792 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.982977    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0df793c-0c7f-468f-b319-1c9c1e1ad934" containerName="main"
Jul 21 17:29:12.985617 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.985601    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:12.986959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.986934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f344ef9c-62eb-478a-b754-e73244aa7069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d476230f-j4d4c\" (UID: \"f344ef9c-62eb-478a-b754-e73244aa7069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:12.988317 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.988301    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:29:12.988410 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.988350    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:29:12.996733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:12.996706    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"]
Jul 21 17:29:13.087904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:13.087868    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f344ef9c-62eb-478a-b754-e73244aa7069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d476230f-j4d4c\" (UID: \"f344ef9c-62eb-478a-b754-e73244aa7069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:13.088505 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:13.088479    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f344ef9c-62eb-478a-b754-e73244aa7069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d476230f-j4d4c\" (UID: \"f344ef9c-62eb-478a-b754-e73244aa7069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:13.295857 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:13.295781    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:13.433234 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:13.433207    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"]
Jul 21 17:29:13.435412 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:29:13.435379    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf344ef9c_62eb_478a_b754_e73244aa7069.slice/crio-d191c5c1721c1f97cb91a950cc4478d4063c3bf9b1e6d4a8550e8f9ed1fba50c WatchSource:0}: Error finding container d191c5c1721c1f97cb91a950cc4478d4063c3bf9b1e6d4a8550e8f9ed1fba50c: Status 404 returned error can't find the container with id d191c5c1721c1f97cb91a950cc4478d4063c3bf9b1e6d4a8550e8f9ed1fba50c
Jul 21 17:29:13.437476 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:13.437455    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:29:13.747504 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:13.747469    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c" event={"ID":"f344ef9c-62eb-478a-b754-e73244aa7069","Type":"ContainerStarted","Data":"d191c5c1721c1f97cb91a950cc4478d4063c3bf9b1e6d4a8550e8f9ed1fba50c"}
Jul 21 17:29:18.778730 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:18.778694    2561 generic.go:356] "Generic (PLEG): container finished" podID="f344ef9c-62eb-478a-b754-e73244aa7069" containerID="233e13f799e52041e8f43418612896f10cc64b30f133ed3a261808673b41a635" exitCode=0
Jul 21 17:29:18.779072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:18.778753    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c" event={"ID":"f344ef9c-62eb-478a-b754-e73244aa7069","Type":"ContainerDied","Data":"233e13f799e52041e8f43418612896f10cc64b30f133ed3a261808673b41a635"}
Jul 21 17:29:19.912143 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:19.912116    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:20.039526 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:20.039440    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f344ef9c-62eb-478a-b754-e73244aa7069-maas-api-service-ca\") pod \"f344ef9c-62eb-478a-b754-e73244aa7069\" (UID: \"f344ef9c-62eb-478a-b754-e73244aa7069\") "
Jul 21 17:29:20.039795 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:20.039773    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f344ef9c-62eb-478a-b754-e73244aa7069-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f344ef9c-62eb-478a-b754-e73244aa7069" (UID: "f344ef9c-62eb-478a-b754-e73244aa7069"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:29:20.140540 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:20.140516    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f344ef9c-62eb-478a-b754-e73244aa7069-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:29:20.790493 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:20.790411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c" event={"ID":"f344ef9c-62eb-478a-b754-e73244aa7069","Type":"ContainerDied","Data":"d191c5c1721c1f97cb91a950cc4478d4063c3bf9b1e6d4a8550e8f9ed1fba50c"}
Jul 21 17:29:20.790493 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:20.790430    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"
Jul 21 17:29:20.790493 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:20.790462    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d191c5c1721c1f97cb91a950cc4478d4063c3bf9b1e6d4a8550e8f9ed1fba50c"
Jul 21 17:29:28.108153 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:28.108113    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"]
Jul 21 17:29:28.113712 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:28.113684    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d476230f-j4d4c"]
Jul 21 17:29:28.513198 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:28.513168    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f344ef9c-62eb-478a-b754-e73244aa7069" path="/var/lib/kubelet/pods/f344ef9c-62eb-478a-b754-e73244aa7069/volumes"
Jul 21 17:29:32.045941 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.045904    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"]
Jul 21 17:29:32.046316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.046252    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f344ef9c-62eb-478a-b754-e73244aa7069" containerName="revoke-keys"
Jul 21 17:29:32.046316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.046264    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f344ef9c-62eb-478a-b754-e73244aa7069" containerName="revoke-keys"
Jul 21 17:29:32.046387 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.046327    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f344ef9c-62eb-478a-b754-e73244aa7069" containerName="revoke-keys"
Jul 21 17:29:32.049276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.049261    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.052044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.052014    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6ac89a3f-serving-cert\""
Jul 21 17:29:32.052174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.052134    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:29:32.052244 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.052183    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:29:32.052967 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.052949    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:29:32.061025 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.060998    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"]
Jul 21 17:29:32.139830 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.139795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8mc8\" (UniqueName: \"kubernetes.io/projected/3e2b901a-8036-4cf5-8544-1d5f75e0e816-kube-api-access-r8mc8\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.139830 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.139833    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2b901a-8036-4cf5-8544-1d5f75e0e816-maas-api-tls\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.140086 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.139866    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2b901a-8036-4cf5-8544-1d5f75e0e816-openshift-service-ca\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.240975 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.240942    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mc8\" (UniqueName: \"kubernetes.io/projected/3e2b901a-8036-4cf5-8544-1d5f75e0e816-kube-api-access-r8mc8\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.240975 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.240985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2b901a-8036-4cf5-8544-1d5f75e0e816-maas-api-tls\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.241250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.241024    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2b901a-8036-4cf5-8544-1d5f75e0e816-openshift-service-ca\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.241770 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.241745    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2b901a-8036-4cf5-8544-1d5f75e0e816-openshift-service-ca\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.243821 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.243799    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2b901a-8036-4cf5-8544-1d5f75e0e816-maas-api-tls\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.250409 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.250363    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8mc8\" (UniqueName: \"kubernetes.io/projected/3e2b901a-8036-4cf5-8544-1d5f75e0e816-kube-api-access-r8mc8\") pod \"maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.362420 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.362340    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:32.505751 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.505718    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"]
Jul 21 17:29:32.510316 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:29:32.510289    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2b901a_8036_4cf5_8544_1d5f75e0e816.slice/crio-6b18326da1d09ef5032d9add47e316d3d887329f10d57ccf5d1bf4fb8504b990 WatchSource:0}: Error finding container 6b18326da1d09ef5032d9add47e316d3d887329f10d57ccf5d1bf4fb8504b990: Status 404 returned error can't find the container with id 6b18326da1d09ef5032d9add47e316d3d887329f10d57ccf5d1bf4fb8504b990
Jul 21 17:29:32.852505 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:32.852461    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c" event={"ID":"3e2b901a-8036-4cf5-8544-1d5f75e0e816","Type":"ContainerStarted","Data":"6b18326da1d09ef5032d9add47e316d3d887329f10d57ccf5d1bf4fb8504b990"}
Jul 21 17:29:34.868507 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:34.868469    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c" event={"ID":"3e2b901a-8036-4cf5-8544-1d5f75e0e816","Type":"ContainerStarted","Data":"2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf"}
Jul 21 17:29:34.869034 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:34.868615    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:34.889470 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:34.889420    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c" podStartSLOduration=1.525471408 podStartE2EDuration="2.889404394s" totalImagesPullingTime="1.363932986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:29:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:29:34.888361993 +0000 UTC m=+1654.853261598" watchObservedRunningTime="2026-07-21 17:29:34.889404394 +0000 UTC m=+1654.854303997"
Jul 21 17:29:40.879255 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:40.879221    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:29:56.728397 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.728357    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"]
Jul 21 17:29:56.736839 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.736817    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:56.739662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.739638    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:29:56.741733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.741698    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"]
Jul 21 17:29:56.855475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.855443    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0accd0-b502-4f6f-b2a7-157347039e5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7\" (UID: \"eb0accd0-b502-4f6f-b2a7-157347039e5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:56.956633 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.956604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0accd0-b502-4f6f-b2a7-157347039e5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7\" (UID: \"eb0accd0-b502-4f6f-b2a7-157347039e5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:56.957179 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:56.957161    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0accd0-b502-4f6f-b2a7-157347039e5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7\" (UID: \"eb0accd0-b502-4f6f-b2a7-157347039e5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:57.048306 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:57.048209    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:57.392543 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:57.392504    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"]
Jul 21 17:29:57.395981 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:29:57.395958    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0accd0_b502_4f6f_b2a7_157347039e5d.slice/crio-c8898e088a7688c86c081cec76b814a081f390bd9f3e01b9c770b19a0c05f6f8 WatchSource:0}: Error finding container c8898e088a7688c86c081cec76b814a081f390bd9f3e01b9c770b19a0c05f6f8: Status 404 returned error can't find the container with id c8898e088a7688c86c081cec76b814a081f390bd9f3e01b9c770b19a0c05f6f8
Jul 21 17:29:57.985292 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:57.985267    2561 generic.go:356] "Generic (PLEG): container finished" podID="eb0accd0-b502-4f6f-b2a7-157347039e5d" containerID="a79fd0370872a4ec8292d7bd3fbb9154ab7da93829a16559a1780357182e3c83" exitCode=0
Jul 21 17:29:57.985615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:57.985349    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7" event={"ID":"eb0accd0-b502-4f6f-b2a7-157347039e5d","Type":"ContainerDied","Data":"a79fd0370872a4ec8292d7bd3fbb9154ab7da93829a16559a1780357182e3c83"}
Jul 21 17:29:57.985615 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:57.985404    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7" event={"ID":"eb0accd0-b502-4f6f-b2a7-157347039e5d","Type":"ContainerStarted","Data":"c8898e088a7688c86c081cec76b814a081f390bd9f3e01b9c770b19a0c05f6f8"}
Jul 21 17:29:59.119398 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.119375    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:59.276652 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.276552    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0accd0-b502-4f6f-b2a7-157347039e5d-maas-api-service-ca\") pod \"eb0accd0-b502-4f6f-b2a7-157347039e5d\" (UID: \"eb0accd0-b502-4f6f-b2a7-157347039e5d\") "
Jul 21 17:29:59.276895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.276875    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb0accd0-b502-4f6f-b2a7-157347039e5d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb0accd0-b502-4f6f-b2a7-157347039e5d" (UID: "eb0accd0-b502-4f6f-b2a7-157347039e5d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:29:59.378146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.378114    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0accd0-b502-4f6f-b2a7-157347039e5d-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:29:59.996759 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.996727    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"
Jul 21 17:29:59.996937 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.996766    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7" event={"ID":"eb0accd0-b502-4f6f-b2a7-157347039e5d","Type":"ContainerDied","Data":"c8898e088a7688c86c081cec76b814a081f390bd9f3e01b9c770b19a0c05f6f8"}
Jul 21 17:29:59.996937 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:29:59.996813    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8898e088a7688c86c081cec76b814a081f390bd9f3e01b9c770b19a0c05f6f8"
Jul 21 17:30:00.147733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.147691    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"]
Jul 21 17:30:00.148097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.148071    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb0accd0-b502-4f6f-b2a7-157347039e5d" containerName="revoke-keys"
Jul 21 17:30:00.148097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.148085    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb0accd0-b502-4f6f-b2a7-157347039e5d" containerName="revoke-keys"
Jul 21 17:30:00.148174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.148152    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb0accd0-b502-4f6f-b2a7-157347039e5d" containerName="revoke-keys"
Jul 21 17:30:00.151416 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.151398    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"]
Jul 21 17:30:00.151535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.151514    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"
Jul 21 17:30:00.154315 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.154290    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:30:00.154805 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.154780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"
Jul 21 17:30:00.159270 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.159014    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"]
Jul 21 17:30:00.163798 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.163728    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"]
Jul 21 17:30:00.166545 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.166518    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"
Jul 21 17:30:00.174535 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.174515    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"
Jul 21 17:30:00.321811 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.321771    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"]
Jul 21 17:30:00.324011 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:30:00.323982    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0621c7ce_5544_4b6f_9460_a9ea5a9a85f0.slice/crio-9a93b986ea562f5ffd95d05b8af527827fc273f9387cc4de28a6c495a190f129 WatchSource:0}: Error finding container 9a93b986ea562f5ffd95d05b8af527827fc273f9387cc4de28a6c495a190f129: Status 404 returned error can't find the container with id 9a93b986ea562f5ffd95d05b8af527827fc273f9387cc4de28a6c495a190f129
Jul 21 17:30:00.347068 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:00.347043    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"]
Jul 21 17:30:00.357665 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:30:00.357638    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027cf3a2_1b54_4c6e_ba45_d473de7e10c7.slice/crio-e1c54e8867eeb544524f19423e28ef817e221b09daea5711d8d49a8c0ab3bf7d WatchSource:0}: Error finding container e1c54e8867eeb544524f19423e28ef817e221b09daea5711d8d49a8c0ab3bf7d: Status 404 returned error can't find the container with id e1c54e8867eeb544524f19423e28ef817e221b09daea5711d8d49a8c0ab3bf7d
Jul 21 17:30:01.003098 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:01.003051    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg" event={"ID":"027cf3a2-1b54-4c6e-ba45-d473de7e10c7","Type":"ContainerStarted","Data":"8255d5fe937d0cc3844fc9a2460606e98fe6bf6a222d13d062efbb5a6a445171"}
Jul 21 17:30:01.003309 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:01.003121    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg" event={"ID":"027cf3a2-1b54-4c6e-ba45-d473de7e10c7","Type":"ContainerStarted","Data":"e1c54e8867eeb544524f19423e28ef817e221b09daea5711d8d49a8c0ab3bf7d"}
Jul 21 17:30:01.004529 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:01.004500    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw" event={"ID":"0621c7ce-5544-4b6f-9460-a9ea5a9a85f0","Type":"ContainerStarted","Data":"6b42dd2f2f12c44ce2f0ebc04ed90dede956062b40468b7e4f3fa189c42782ce"}
Jul 21 17:30:01.004678 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:01.004545    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw" event={"ID":"0621c7ce-5544-4b6f-9460-a9ea5a9a85f0","Type":"ContainerStarted","Data":"9a93b986ea562f5ffd95d05b8af527827fc273f9387cc4de28a6c495a190f129"}
Jul 21 17:30:01.021454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:01.021408    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg" podStartSLOduration=1.021393661 podStartE2EDuration="1.021393661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:30:01.020226265 +0000 UTC m=+1680.985125866" watchObservedRunningTime="2026-07-21 17:30:01.021393661 +0000 UTC m=+1680.986293334"
Jul 21 17:30:01.045331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:01.045285    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw" podStartSLOduration=1.045270984 podStartE2EDuration="1.045270984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:30:01.043361669 +0000 UTC m=+1681.008261274" watchObservedRunningTime="2026-07-21 17:30:01.045270984 +0000 UTC m=+1681.010170587"
Jul 21 17:30:02.011286 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:02.011248    2561 generic.go:356] "Generic (PLEG): container finished" podID="027cf3a2-1b54-4c6e-ba45-d473de7e10c7" containerID="8255d5fe937d0cc3844fc9a2460606e98fe6bf6a222d13d062efbb5a6a445171" exitCode=0
Jul 21 17:30:02.011786 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:02.011336    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg" event={"ID":"027cf3a2-1b54-4c6e-ba45-d473de7e10c7","Type":"ContainerDied","Data":"8255d5fe937d0cc3844fc9a2460606e98fe6bf6a222d13d062efbb5a6a445171"}
Jul 21 17:30:02.012894 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:02.012873    2561 generic.go:356] "Generic (PLEG): container finished" podID="0621c7ce-5544-4b6f-9460-a9ea5a9a85f0" containerID="6b42dd2f2f12c44ce2f0ebc04ed90dede956062b40468b7e4f3fa189c42782ce" exitCode=0
Jul 21 17:30:02.013004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:02.012945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw" event={"ID":"0621c7ce-5544-4b6f-9460-a9ea5a9a85f0","Type":"ContainerDied","Data":"6b42dd2f2f12c44ce2f0ebc04ed90dede956062b40468b7e4f3fa189c42782ce"}
Jul 21 17:30:03.161471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:03.161447    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"
Jul 21 17:30:03.187656 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:03.187631    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"
Jul 21 17:30:04.024840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:04.024812    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"
Jul 21 17:30:04.025017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:04.024853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg" event={"ID":"027cf3a2-1b54-4c6e-ba45-d473de7e10c7","Type":"ContainerDied","Data":"e1c54e8867eeb544524f19423e28ef817e221b09daea5711d8d49a8c0ab3bf7d"}
Jul 21 17:30:04.025017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:04.024895    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1c54e8867eeb544524f19423e28ef817e221b09daea5711d8d49a8c0ab3bf7d"
Jul 21 17:30:04.026061 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:04.026040    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw"
Jul 21 17:30:04.026061 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:04.026048    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744250-fvxdw" event={"ID":"0621c7ce-5544-4b6f-9460-a9ea5a9a85f0","Type":"ContainerDied","Data":"9a93b986ea562f5ffd95d05b8af527827fc273f9387cc4de28a6c495a190f129"}
Jul 21 17:30:04.026206 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:04.026088    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a93b986ea562f5ffd95d05b8af527827fc273f9387cc4de28a6c495a190f129"
Jul 21 17:30:07.254085 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.254052    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"]
Jul 21 17:30:07.255075 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.255014    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c" podUID="3e2b901a-8036-4cf5-8544-1d5f75e0e816" containerName="maas-api" containerID="cri-o://2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf" gracePeriod=30
Jul 21 17:30:07.291500 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.291471    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"]
Jul 21 17:30:07.295631 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.295602    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-6ac89a3f-29744250-7j8dg"]
Jul 21 17:30:07.508529 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.508476    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:30:07.653276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.653239    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8mc8\" (UniqueName: \"kubernetes.io/projected/3e2b901a-8036-4cf5-8544-1d5f75e0e816-kube-api-access-r8mc8\") pod \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") "
Jul 21 17:30:07.653448 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.653306    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2b901a-8036-4cf5-8544-1d5f75e0e816-openshift-service-ca\") pod \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") "
Jul 21 17:30:07.653448 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.653347    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2b901a-8036-4cf5-8544-1d5f75e0e816-maas-api-tls\") pod \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\" (UID: \"3e2b901a-8036-4cf5-8544-1d5f75e0e816\") "
Jul 21 17:30:07.653775 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.653744    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2b901a-8036-4cf5-8544-1d5f75e0e816-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e2b901a-8036-4cf5-8544-1d5f75e0e816" (UID: "3e2b901a-8036-4cf5-8544-1d5f75e0e816"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:30:07.655771 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.655740    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2b901a-8036-4cf5-8544-1d5f75e0e816-kube-api-access-r8mc8" (OuterVolumeSpecName: "kube-api-access-r8mc8") pod "3e2b901a-8036-4cf5-8544-1d5f75e0e816" (UID: "3e2b901a-8036-4cf5-8544-1d5f75e0e816"). InnerVolumeSpecName "kube-api-access-r8mc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:30:07.655885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.655832    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2b901a-8036-4cf5-8544-1d5f75e0e816-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e2b901a-8036-4cf5-8544-1d5f75e0e816" (UID: "3e2b901a-8036-4cf5-8544-1d5f75e0e816"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:30:07.753999 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.753973    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8mc8\" (UniqueName: \"kubernetes.io/projected/3e2b901a-8036-4cf5-8544-1d5f75e0e816-kube-api-access-r8mc8\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:07.753999 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.753998    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2b901a-8036-4cf5-8544-1d5f75e0e816-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:07.754193 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:07.754008    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e2b901a-8036-4cf5-8544-1d5f75e0e816-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:08.048996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.048964    2561 generic.go:356] "Generic (PLEG): container finished" podID="3e2b901a-8036-4cf5-8544-1d5f75e0e816" containerID="2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf" exitCode=0
Jul 21 17:30:08.049190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.049018    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"
Jul 21 17:30:08.049190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.049059    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c" event={"ID":"3e2b901a-8036-4cf5-8544-1d5f75e0e816","Type":"ContainerDied","Data":"2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf"}
Jul 21 17:30:08.049190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.049124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c" event={"ID":"3e2b901a-8036-4cf5-8544-1d5f75e0e816","Type":"ContainerDied","Data":"6b18326da1d09ef5032d9add47e316d3d887329f10d57ccf5d1bf4fb8504b990"}
Jul 21 17:30:08.049190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.049157    2561 scope.go:117] "RemoveContainer" containerID="2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf"
Jul 21 17:30:08.059967 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.059951    2561 scope.go:117] "RemoveContainer" containerID="2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf"
Jul 21 17:30:08.060226 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:30:08.060208    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf\": container with ID starting with 2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf not found: ID does not exist" containerID="2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf"
Jul 21 17:30:08.060290 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.060233    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf"} err="failed to get container status \"2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf\": rpc error: code = NotFound desc = could not find container \"2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf\": container with ID starting with 2dcbc108e4963eb6584d66653e8d6ffc33dd462a17aaa866c2f8dfb76a6080cf not found: ID does not exist"
Jul 21 17:30:08.074228 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.074206    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"]
Jul 21 17:30:08.080097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.080073    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ac89a3f-584fcdddc7-g2d6c"]
Jul 21 17:30:08.514185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.514141    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="027cf3a2-1b54-4c6e-ba45-d473de7e10c7" path="/var/lib/kubelet/pods/027cf3a2-1b54-4c6e-ba45-d473de7e10c7/volumes"
Jul 21 17:30:08.514594 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:08.514547    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e2b901a-8036-4cf5-8544-1d5f75e0e816" path="/var/lib/kubelet/pods/3e2b901a-8036-4cf5-8544-1d5f75e0e816/volumes"
Jul 21 17:30:11.829573 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:11.829529    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"]
Jul 21 17:30:11.834821 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:11.834794    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ac89a3f-h55l7"]
Jul 21 17:30:12.513418 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:12.513384    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb0accd0-b502-4f6f-b2a7-157347039e5d" path="/var/lib/kubelet/pods/eb0accd0-b502-4f6f-b2a7-157347039e5d/volumes"
Jul 21 17:30:15.323905 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.323875    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"]
Jul 21 17:30:15.324262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324212    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027cf3a2-1b54-4c6e-ba45-d473de7e10c7" containerName="cleanup"
Jul 21 17:30:15.324262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324224    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="027cf3a2-1b54-4c6e-ba45-d473de7e10c7" containerName="cleanup"
Jul 21 17:30:15.324262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324246    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e2b901a-8036-4cf5-8544-1d5f75e0e816" containerName="maas-api"
Jul 21 17:30:15.324262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324252    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e2b901a-8036-4cf5-8544-1d5f75e0e816" containerName="maas-api"
Jul 21 17:30:15.324262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324260    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0621c7ce-5544-4b6f-9460-a9ea5a9a85f0" containerName="cleanup"
Jul 21 17:30:15.324262 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324266    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0621c7ce-5544-4b6f-9460-a9ea5a9a85f0" containerName="cleanup"
Jul 21 17:30:15.324451 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324320    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e2b901a-8036-4cf5-8544-1d5f75e0e816" containerName="maas-api"
Jul 21 17:30:15.324451 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324332    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0621c7ce-5544-4b6f-9460-a9ea5a9a85f0" containerName="cleanup"
Jul 21 17:30:15.324451 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.324342    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="027cf3a2-1b54-4c6e-ba45-d473de7e10c7" containerName="cleanup"
Jul 21 17:30:15.328679 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.328664    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.331514 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.331493    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4db918c7-serving-cert\""
Jul 21 17:30:15.331662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.331612    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:30:15.331728 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.331712    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:30:15.332313 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.332298    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:30:15.343648 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.343625    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"]
Jul 21 17:30:15.421334 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.421305    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.421334 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.421343    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3772e270-f756-4ebd-91ca-94566bdf350a-openshift-service-ca\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.421546 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.421363    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8gzk\" (UniqueName: \"kubernetes.io/projected/3772e270-f756-4ebd-91ca-94566bdf350a-kube-api-access-k8gzk\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.521839 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.521806    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.522004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.521846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3772e270-f756-4ebd-91ca-94566bdf350a-openshift-service-ca\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.522004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.521867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8gzk\" (UniqueName: \"kubernetes.io/projected/3772e270-f756-4ebd-91ca-94566bdf350a-kube-api-access-k8gzk\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.522004 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:30:15.521954    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-serving-cert: secret "maas-api-e2e-ait-4db918c7-serving-cert" not found
Jul 21 17:30:15.522139 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:30:15.522031    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls podName:3772e270-f756-4ebd-91ca-94566bdf350a nodeName:}" failed. No retries permitted until 2026-07-21 17:30:16.022012822 +0000 UTC m=+1695.986912404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls") pod "maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" (UID: "3772e270-f756-4ebd-91ca-94566bdf350a") : secret "maas-api-e2e-ait-4db918c7-serving-cert" not found
Jul 21 17:30:15.522521 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.522504    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3772e270-f756-4ebd-91ca-94566bdf350a-openshift-service-ca\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:15.532885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:15.532859    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8gzk\" (UniqueName: \"kubernetes.io/projected/3772e270-f756-4ebd-91ca-94566bdf350a-kube-api-access-k8gzk\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:16.025142 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:16.025100    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:16.027831 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:16.027806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls\") pod \"maas-api-e2e-ait-4db918c7-6bf5bf547-lw476\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:16.238687 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:16.238657    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:16.377655 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:16.377631    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"]
Jul 21 17:30:16.379426 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:30:16.379396    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3772e270_f756_4ebd_91ca_94566bdf350a.slice/crio-b3e98854cab3187ddc2d43ccf7455ff8669f5018cd24705b26c2a9b5e9b46c34 WatchSource:0}: Error finding container b3e98854cab3187ddc2d43ccf7455ff8669f5018cd24705b26c2a9b5e9b46c34: Status 404 returned error can't find the container with id b3e98854cab3187ddc2d43ccf7455ff8669f5018cd24705b26c2a9b5e9b46c34
Jul 21 17:30:17.104940 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:17.104846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" event={"ID":"3772e270-f756-4ebd-91ca-94566bdf350a","Type":"ContainerStarted","Data":"118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf"}
Jul 21 17:30:17.104940 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:17.104903    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" event={"ID":"3772e270-f756-4ebd-91ca-94566bdf350a","Type":"ContainerStarted","Data":"b3e98854cab3187ddc2d43ccf7455ff8669f5018cd24705b26c2a9b5e9b46c34"}
Jul 21 17:30:17.104940 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:17.104923    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:17.143221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:17.143173    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" podStartSLOduration=1.7622720269999999 podStartE2EDuration="2.143158408s" totalImagesPullingTime="380.886381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:30:17.140551163 +0000 UTC m=+1697.105450769" watchObservedRunningTime="2026-07-21 17:30:17.143158408 +0000 UTC m=+1697.108058024"
Jul 21 17:30:23.117056 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:23.116977    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:40.745399 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.745364    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"]
Jul 21 17:30:40.748857 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.748840    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:40.751401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.751382    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:30:40.759231 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.759206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"]
Jul 21 17:30:40.829742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.829704    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9045b77-10df-42ff-bedd-13ab1239e3c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5\" (UID: \"e9045b77-10df-42ff-bedd-13ab1239e3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:40.930830 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.930792    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9045b77-10df-42ff-bedd-13ab1239e3c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5\" (UID: \"e9045b77-10df-42ff-bedd-13ab1239e3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:40.931384 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:40.931364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9045b77-10df-42ff-bedd-13ab1239e3c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5\" (UID: \"e9045b77-10df-42ff-bedd-13ab1239e3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:41.060855 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:41.060770    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:41.404463 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:41.404438    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"]
Jul 21 17:30:41.406582 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:30:41.406538    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9045b77_10df_42ff_bedd_13ab1239e3c6.slice/crio-c4aa8634ddcc3e92511dac1764e47838810a0e8ab49c3741d8635f30f7f1c779 WatchSource:0}: Error finding container c4aa8634ddcc3e92511dac1764e47838810a0e8ab49c3741d8635f30f7f1c779: Status 404 returned error can't find the container with id c4aa8634ddcc3e92511dac1764e47838810a0e8ab49c3741d8635f30f7f1c779
Jul 21 17:30:42.244292 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:42.244259    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9045b77-10df-42ff-bedd-13ab1239e3c6" containerID="84ee8a47fde8bea4c4c5d58bd9d9f5da2b693bda0acffc3306b7040f072a7c17" exitCode=0
Jul 21 17:30:42.244706 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:42.244309    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5" event={"ID":"e9045b77-10df-42ff-bedd-13ab1239e3c6","Type":"ContainerDied","Data":"84ee8a47fde8bea4c4c5d58bd9d9f5da2b693bda0acffc3306b7040f072a7c17"}
Jul 21 17:30:42.244706 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:42.244359    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5" event={"ID":"e9045b77-10df-42ff-bedd-13ab1239e3c6","Type":"ContainerStarted","Data":"c4aa8634ddcc3e92511dac1764e47838810a0e8ab49c3741d8635f30f7f1c779"}
Jul 21 17:30:43.374902 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:43.374872    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:43.452837 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:43.452799    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9045b77-10df-42ff-bedd-13ab1239e3c6-maas-api-service-ca\") pod \"e9045b77-10df-42ff-bedd-13ab1239e3c6\" (UID: \"e9045b77-10df-42ff-bedd-13ab1239e3c6\") "
Jul 21 17:30:43.453168 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:43.453145    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9045b77-10df-42ff-bedd-13ab1239e3c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9045b77-10df-42ff-bedd-13ab1239e3c6" (UID: "e9045b77-10df-42ff-bedd-13ab1239e3c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:30:43.554513 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:43.554428    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9045b77-10df-42ff-bedd-13ab1239e3c6-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:44.255995 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:44.255962    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"
Jul 21 17:30:44.255995 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:44.255975    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5" event={"ID":"e9045b77-10df-42ff-bedd-13ab1239e3c6","Type":"ContainerDied","Data":"c4aa8634ddcc3e92511dac1764e47838810a0e8ab49c3741d8635f30f7f1c779"}
Jul 21 17:30:44.256245 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:44.256023    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4aa8634ddcc3e92511dac1764e47838810a0e8ab49c3741d8635f30f7f1c779"
Jul 21 17:30:51.328482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.327792    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"]
Jul 21 17:30:51.328482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.328227    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" podUID="3772e270-f756-4ebd-91ca-94566bdf350a" containerName="maas-api" containerID="cri-o://118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf" gracePeriod=30
Jul 21 17:30:51.580549 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.580494    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:51.732545 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.732517    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3772e270-f756-4ebd-91ca-94566bdf350a-openshift-service-ca\") pod \"3772e270-f756-4ebd-91ca-94566bdf350a\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") "
Jul 21 17:30:51.732732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.732635    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8gzk\" (UniqueName: \"kubernetes.io/projected/3772e270-f756-4ebd-91ca-94566bdf350a-kube-api-access-k8gzk\") pod \"3772e270-f756-4ebd-91ca-94566bdf350a\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") "
Jul 21 17:30:51.732732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.732667    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls\") pod \"3772e270-f756-4ebd-91ca-94566bdf350a\" (UID: \"3772e270-f756-4ebd-91ca-94566bdf350a\") "
Jul 21 17:30:51.732927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.732899    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3772e270-f756-4ebd-91ca-94566bdf350a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3772e270-f756-4ebd-91ca-94566bdf350a" (UID: "3772e270-f756-4ebd-91ca-94566bdf350a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:30:51.734664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.734633    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3772e270-f756-4ebd-91ca-94566bdf350a-kube-api-access-k8gzk" (OuterVolumeSpecName: "kube-api-access-k8gzk") pod "3772e270-f756-4ebd-91ca-94566bdf350a" (UID: "3772e270-f756-4ebd-91ca-94566bdf350a"). InnerVolumeSpecName "kube-api-access-k8gzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:30:51.734769 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.734743    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3772e270-f756-4ebd-91ca-94566bdf350a" (UID: "3772e270-f756-4ebd-91ca-94566bdf350a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:30:51.833961 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.833890    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8gzk\" (UniqueName: \"kubernetes.io/projected/3772e270-f756-4ebd-91ca-94566bdf350a-kube-api-access-k8gzk\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:51.833961 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.833916    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3772e270-f756-4ebd-91ca-94566bdf350a-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:51.833961 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:51.833926    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3772e270-f756-4ebd-91ca-94566bdf350a-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:30:52.301752 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.301709    2561 generic.go:356] "Generic (PLEG): container finished" podID="3772e270-f756-4ebd-91ca-94566bdf350a" containerID="118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf" exitCode=0
Jul 21 17:30:52.301956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.301755    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"
Jul 21 17:30:52.301956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.301803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" event={"ID":"3772e270-f756-4ebd-91ca-94566bdf350a","Type":"ContainerDied","Data":"118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf"}
Jul 21 17:30:52.301956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.301862    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476" event={"ID":"3772e270-f756-4ebd-91ca-94566bdf350a","Type":"ContainerDied","Data":"b3e98854cab3187ddc2d43ccf7455ff8669f5018cd24705b26c2a9b5e9b46c34"}
Jul 21 17:30:52.301956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.301892    2561 scope.go:117] "RemoveContainer" containerID="118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf"
Jul 21 17:30:52.314607 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.314590    2561 scope.go:117] "RemoveContainer" containerID="118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf"
Jul 21 17:30:52.314895 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:30:52.314873    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf\": container with ID starting with 118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf not found: ID does not exist" containerID="118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf"
Jul 21 17:30:52.314967 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.314908    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf"} err="failed to get container status \"118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf\": rpc error: code = NotFound desc = could not find container \"118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf\": container with ID starting with 118d9b4b02af62591f90ee201fcd9a7bcc11ae0cef794d4b8b2bc2af141822bf not found: ID does not exist"
Jul 21 17:30:52.329869 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.329841    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"]
Jul 21 17:30:52.335948 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.335922    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4db918c7-6bf5bf547-lw476"]
Jul 21 17:30:52.514260 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:52.514231    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3772e270-f756-4ebd-91ca-94566bdf350a" path="/var/lib/kubelet/pods/3772e270-f756-4ebd-91ca-94566bdf350a/volumes"
Jul 21 17:30:55.851460 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:55.851419    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"]
Jul 21 17:30:55.855360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:55.855333    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4db918c7-q7sh5"]
Jul 21 17:30:56.513818 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:30:56.513784    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9045b77-10df-42ff-bedd-13ab1239e3c6" path="/var/lib/kubelet/pods/e9045b77-10df-42ff-bedd-13ab1239e3c6/volumes"
Jul 21 17:31:00.260846 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.260800    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"]
Jul 21 17:31:00.261347 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.261323    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3772e270-f756-4ebd-91ca-94566bdf350a" containerName="maas-api"
Jul 21 17:31:00.261417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.261352    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3772e270-f756-4ebd-91ca-94566bdf350a" containerName="maas-api"
Jul 21 17:31:00.261417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.261368    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9045b77-10df-42ff-bedd-13ab1239e3c6" containerName="revoke-keys"
Jul 21 17:31:00.261417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.261379    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9045b77-10df-42ff-bedd-13ab1239e3c6" containerName="revoke-keys"
Jul 21 17:31:00.261593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.261470    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9045b77-10df-42ff-bedd-13ab1239e3c6" containerName="revoke-keys"
Jul 21 17:31:00.261593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.261484    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3772e270-f756-4ebd-91ca-94566bdf350a" containerName="maas-api"
Jul 21 17:31:00.266148 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.266124    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.268439 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.268416    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:31:00.268808 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.268788    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2ca71765-serving-cert\""
Jul 21 17:31:00.269311 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.269292    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:31:00.269424 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.269293    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:31:00.276292 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.276267    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"]
Jul 21 17:31:00.412890 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.412853    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22h27\" (UniqueName: \"kubernetes.io/projected/4698047e-8f1a-4148-980b-b0745e50d883-kube-api-access-22h27\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.412890 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.412905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4698047e-8f1a-4148-980b-b0745e50d883-openshift-service-ca\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.413180 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.413078    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4698047e-8f1a-4148-980b-b0745e50d883-maas-api-tls\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.513985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.513902    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22h27\" (UniqueName: \"kubernetes.io/projected/4698047e-8f1a-4148-980b-b0745e50d883-kube-api-access-22h27\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.513985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.513946    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4698047e-8f1a-4148-980b-b0745e50d883-openshift-service-ca\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.514196 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.514011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4698047e-8f1a-4148-980b-b0745e50d883-maas-api-tls\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.518518 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.518497    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:31:00.518658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.518517    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2ca71765-serving-cert\""
Jul 21 17:31:00.525109 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.525091    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:31:00.525423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.525402    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4698047e-8f1a-4148-980b-b0745e50d883-openshift-service-ca\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.527154 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.527133    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4698047e-8f1a-4148-980b-b0745e50d883-maas-api-tls\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.535731 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.535707    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22h27\" (UniqueName: \"kubernetes.io/projected/4698047e-8f1a-4148-980b-b0745e50d883-kube-api-access-22h27\") pod \"maas-api-e2e-derive-2ca71765-565cb8844-hwg9x\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.580627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.580596    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:31:00.588636 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.588611    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:00.734127 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:00.734091    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"]
Jul 21 17:31:00.736337 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:31:00.736310    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4698047e_8f1a_4148_980b_b0745e50d883.slice/crio-0a4f95c49dde73bf3db67624f99c93cd464ee53f1d373290edeb6640134221b2 WatchSource:0}: Error finding container 0a4f95c49dde73bf3db67624f99c93cd464ee53f1d373290edeb6640134221b2: Status 404 returned error can't find the container with id 0a4f95c49dde73bf3db67624f99c93cd464ee53f1d373290edeb6640134221b2
Jul 21 17:31:01.352528 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:01.352446    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x" event={"ID":"4698047e-8f1a-4148-980b-b0745e50d883","Type":"ContainerStarted","Data":"0a4f95c49dde73bf3db67624f99c93cd464ee53f1d373290edeb6640134221b2"}
Jul 21 17:31:02.362364 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:02.362325    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x" event={"ID":"4698047e-8f1a-4148-980b-b0745e50d883","Type":"ContainerStarted","Data":"1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8"}
Jul 21 17:31:02.362853 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:02.362443    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:02.386759 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:02.386710    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x" podStartSLOduration=1.712757058 podStartE2EDuration="2.386693094s" totalImagesPullingTime="673.936036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:31:02.386585657 +0000 UTC m=+1742.351485257" watchObservedRunningTime="2026-07-21 17:31:02.386693094 +0000 UTC m=+1742.351592703"
Jul 21 17:31:08.372762 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:08.372729    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:25.048987 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.048947    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"]
Jul 21 17:31:25.052483 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.052466    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:25.055041 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.055024    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:31:25.061756 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.061733    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"]
Jul 21 17:31:25.121655 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.121617    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj\" (UID: \"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:25.222102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.222072    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj\" (UID: \"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:25.222742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.222723    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj\" (UID: \"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:25.364544 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.364458    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:25.512551 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:25.512527    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"]
Jul 21 17:31:25.513924 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:31:25.513901    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffda3b04_9094_48c0_8bd3_cf8c1ba1bd74.slice/crio-c01656cc314d4a45ff660978aa612e1c9209c28035f9f0b53fc762916cd12c68 WatchSource:0}: Error finding container c01656cc314d4a45ff660978aa612e1c9209c28035f9f0b53fc762916cd12c68: Status 404 returned error can't find the container with id c01656cc314d4a45ff660978aa612e1c9209c28035f9f0b53fc762916cd12c68
Jul 21 17:31:26.499068 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:26.499030    2561 generic.go:356] "Generic (PLEG): container finished" podID="ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74" containerID="23747ac04141a6260a5f783fae562d5088a61a2183d1c4d654b20cc45ad80367" exitCode=0
Jul 21 17:31:26.499440 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:26.499119    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj" event={"ID":"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74","Type":"ContainerDied","Data":"23747ac04141a6260a5f783fae562d5088a61a2183d1c4d654b20cc45ad80367"}
Jul 21 17:31:26.499440 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:26.499187    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj" event={"ID":"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74","Type":"ContainerStarted","Data":"c01656cc314d4a45ff660978aa612e1c9209c28035f9f0b53fc762916cd12c68"}
Jul 21 17:31:27.638456 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:27.638433    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:27.743373 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:27.743342    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74-maas-api-service-ca\") pod \"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74\" (UID: \"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74\") "
Jul 21 17:31:27.743744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:27.743719    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74" (UID: "ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:31:27.844331 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:27.844258    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:31:28.511260 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:28.511229    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"
Jul 21 17:31:28.513571 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:28.513527    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj" event={"ID":"ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74","Type":"ContainerDied","Data":"c01656cc314d4a45ff660978aa612e1c9209c28035f9f0b53fc762916cd12c68"}
Jul 21 17:31:28.513693 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:28.513596    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c01656cc314d4a45ff660978aa612e1c9209c28035f9f0b53fc762916cd12c68"
Jul 21 17:31:35.553256 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.553224    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"]
Jul 21 17:31:35.553750 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.553477    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x" podUID="4698047e-8f1a-4148-980b-b0745e50d883" containerName="maas-api" containerID="cri-o://1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8" gracePeriod=30
Jul 21 17:31:35.808015 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.807953    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:35.913342 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.913318    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4698047e-8f1a-4148-980b-b0745e50d883-openshift-service-ca\") pod \"4698047e-8f1a-4148-980b-b0745e50d883\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") "
Jul 21 17:31:35.913479 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.913392    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22h27\" (UniqueName: \"kubernetes.io/projected/4698047e-8f1a-4148-980b-b0745e50d883-kube-api-access-22h27\") pod \"4698047e-8f1a-4148-980b-b0745e50d883\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") "
Jul 21 17:31:35.913523 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.913489    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4698047e-8f1a-4148-980b-b0745e50d883-maas-api-tls\") pod \"4698047e-8f1a-4148-980b-b0745e50d883\" (UID: \"4698047e-8f1a-4148-980b-b0745e50d883\") "
Jul 21 17:31:35.913726 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.913703    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4698047e-8f1a-4148-980b-b0745e50d883-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4698047e-8f1a-4148-980b-b0745e50d883" (UID: "4698047e-8f1a-4148-980b-b0745e50d883"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:31:35.915610 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.915580    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4698047e-8f1a-4148-980b-b0745e50d883-kube-api-access-22h27" (OuterVolumeSpecName: "kube-api-access-22h27") pod "4698047e-8f1a-4148-980b-b0745e50d883" (UID: "4698047e-8f1a-4148-980b-b0745e50d883"). InnerVolumeSpecName "kube-api-access-22h27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:31:35.915610 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:35.915599    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4698047e-8f1a-4148-980b-b0745e50d883-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4698047e-8f1a-4148-980b-b0745e50d883" (UID: "4698047e-8f1a-4148-980b-b0745e50d883"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:31:36.014832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.014797    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4698047e-8f1a-4148-980b-b0745e50d883-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:31:36.014832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.014827    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22h27\" (UniqueName: \"kubernetes.io/projected/4698047e-8f1a-4148-980b-b0745e50d883-kube-api-access-22h27\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:31:36.015013 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.014840    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4698047e-8f1a-4148-980b-b0745e50d883-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:31:36.556976 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.556943    2561 generic.go:356] "Generic (PLEG): container finished" podID="4698047e-8f1a-4148-980b-b0745e50d883" containerID="1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8" exitCode=0
Jul 21 17:31:36.557415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.557004    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"
Jul 21 17:31:36.557415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.557044    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x" event={"ID":"4698047e-8f1a-4148-980b-b0745e50d883","Type":"ContainerDied","Data":"1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8"}
Jul 21 17:31:36.557415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.557095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x" event={"ID":"4698047e-8f1a-4148-980b-b0745e50d883","Type":"ContainerDied","Data":"0a4f95c49dde73bf3db67624f99c93cd464ee53f1d373290edeb6640134221b2"}
Jul 21 17:31:36.557415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.557119    2561 scope.go:117] "RemoveContainer" containerID="1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8"
Jul 21 17:31:36.566621 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.566591    2561 scope.go:117] "RemoveContainer" containerID="1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8"
Jul 21 17:31:36.566901 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:31:36.566877    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8\": container with ID starting with 1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8 not found: ID does not exist" containerID="1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8"
Jul 21 17:31:36.566982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.566912    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8"} err="failed to get container status \"1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8\": rpc error: code = NotFound desc = could not find container \"1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8\": container with ID starting with 1994d333b3f7497a5c908db4c072cd005560004ddb79c1de266c49e2f25fbec8 not found: ID does not exist"
Jul 21 17:31:36.575752 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.575728    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"]
Jul 21 17:31:36.581205 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:36.581184    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2ca71765-565cb8844-hwg9x"]
Jul 21 17:31:38.513990 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:38.513955    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4698047e-8f1a-4148-980b-b0745e50d883" path="/var/lib/kubelet/pods/4698047e-8f1a-4148-980b-b0745e50d883/volumes"
Jul 21 17:31:40.149827 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:40.149789    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"]
Jul 21 17:31:40.151922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:40.151900    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2ca71765-2qnkj"]
Jul 21 17:31:40.515425 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:31:40.515347    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74" path="/var/lib/kubelet/pods/ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74/volumes"
Jul 21 17:32:00.601032 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:32:00.601005    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:32:00.603627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:32:00.603607    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:33:29.707700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.707623    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"]
Jul 21 17:33:29.708141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.707979    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74" containerName="revoke-keys"
Jul 21 17:33:29.708141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.707991    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74" containerName="revoke-keys"
Jul 21 17:33:29.708141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.708001    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4698047e-8f1a-4148-980b-b0745e50d883" containerName="maas-api"
Jul 21 17:33:29.708141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.708006    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4698047e-8f1a-4148-980b-b0745e50d883" containerName="maas-api"
Jul 21 17:33:29.708141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.708077    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4698047e-8f1a-4148-980b-b0745e50d883" containerName="maas-api"
Jul 21 17:33:29.708141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.708089    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffda3b04-9094-48c0-8bd3-cf8c1ba1bd74" containerName="revoke-keys"
Jul 21 17:33:29.711103 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.711081    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.713413 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.713388    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d01307af-serving-cert\""
Jul 21 17:33:29.713868 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.713843    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:33:29.714298 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.714284    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:33:29.715040 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.715021    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:33:29.723665 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.723641    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"]
Jul 21 17:33:29.887316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.887280    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9e197ea-35c5-4f14-915c-37af9d6a1bed-maas-api-tls\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.887316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.887318    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e197ea-35c5-4f14-915c-37af9d6a1bed-openshift-service-ca\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.887546 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.887345    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52h29\" (UniqueName: \"kubernetes.io/projected/d9e197ea-35c5-4f14-915c-37af9d6a1bed-kube-api-access-52h29\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.987907 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.987820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9e197ea-35c5-4f14-915c-37af9d6a1bed-maas-api-tls\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.987907 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.987854    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e197ea-35c5-4f14-915c-37af9d6a1bed-openshift-service-ca\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.987907 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.987882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52h29\" (UniqueName: \"kubernetes.io/projected/d9e197ea-35c5-4f14-915c-37af9d6a1bed-kube-api-access-52h29\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.988458 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.988430    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e197ea-35c5-4f14-915c-37af9d6a1bed-openshift-service-ca\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.990375 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.990353    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9e197ea-35c5-4f14-915c-37af9d6a1bed-maas-api-tls\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:29.998865 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:29.998836    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52h29\" (UniqueName: \"kubernetes.io/projected/d9e197ea-35c5-4f14-915c-37af9d6a1bed-kube-api-access-52h29\") pod \"maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:30.021759 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:30.021729    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:30.162016 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:30.161985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"]
Jul 21 17:33:30.163768 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:33:30.163740    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e197ea_35c5_4f14_915c_37af9d6a1bed.slice/crio-128b6a324185e16613abcef7e6394891c3b1c6d3e66bb9297680b7ed311cbda2 WatchSource:0}: Error finding container 128b6a324185e16613abcef7e6394891c3b1c6d3e66bb9297680b7ed311cbda2: Status 404 returned error can't find the container with id 128b6a324185e16613abcef7e6394891c3b1c6d3e66bb9297680b7ed311cbda2
Jul 21 17:33:31.174806 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:31.174768    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q" event={"ID":"d9e197ea-35c5-4f14-915c-37af9d6a1bed","Type":"ContainerStarted","Data":"47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c"}
Jul 21 17:33:31.175272 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:31.174828    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q" event={"ID":"d9e197ea-35c5-4f14-915c-37af9d6a1bed","Type":"ContainerStarted","Data":"128b6a324185e16613abcef7e6394891c3b1c6d3e66bb9297680b7ed311cbda2"}
Jul 21 17:33:31.175272 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:31.174906    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:31.193957 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:31.193903    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q" podStartSLOduration=1.808027788 podStartE2EDuration="2.193886848s" totalImagesPullingTime="385.85906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:33:31.193544539 +0000 UTC m=+1891.158444191" watchObservedRunningTime="2026-07-21 17:33:31.193886848 +0000 UTC m=+1891.158786454"
Jul 21 17:33:37.185839 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:37.185811    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:42.003141 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.003093    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"]
Jul 21 17:33:42.006749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.006726    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:42.010311 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.010278    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:33:42.018126 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.018103    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"]
Jul 21 17:33:42.097457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.097415    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e6de2e-d73e-42dc-a316-e5f552c27d0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d01307af-bbbtz\" (UID: \"48e6de2e-d73e-42dc-a316-e5f552c27d0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:42.198576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.198527    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e6de2e-d73e-42dc-a316-e5f552c27d0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d01307af-bbbtz\" (UID: \"48e6de2e-d73e-42dc-a316-e5f552c27d0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:42.199129 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.199108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e6de2e-d73e-42dc-a316-e5f552c27d0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d01307af-bbbtz\" (UID: \"48e6de2e-d73e-42dc-a316-e5f552c27d0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:42.318703 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.318617    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:42.453999 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:42.453968    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"]
Jul 21 17:33:42.455006 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:33:42.454978    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e6de2e_d73e_42dc_a316_e5f552c27d0d.slice/crio-094bac215f3c1cc2064b2fa22c760bc82d72c98c26576843f9b1f8407cd763e4 WatchSource:0}: Error finding container 094bac215f3c1cc2064b2fa22c760bc82d72c98c26576843f9b1f8407cd763e4: Status 404 returned error can't find the container with id 094bac215f3c1cc2064b2fa22c760bc82d72c98c26576843f9b1f8407cd763e4
Jul 21 17:33:43.247105 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:43.247068    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz" event={"ID":"48e6de2e-d73e-42dc-a316-e5f552c27d0d","Type":"ContainerStarted","Data":"65d08149213dc6f6ab08c09af0ad64bc3b39d1c3b9eeacc1dc84a088827a3f39"}
Jul 21 17:33:43.247493 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:43.247117    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz" event={"ID":"48e6de2e-d73e-42dc-a316-e5f552c27d0d","Type":"ContainerStarted","Data":"094bac215f3c1cc2064b2fa22c760bc82d72c98c26576843f9b1f8407cd763e4"}
Jul 21 17:33:43.268638 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:43.268545    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz" podStartSLOduration=2.268531142 podStartE2EDuration="2.268531142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:33:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:33:43.265993156 +0000 UTC m=+1903.230892761" watchObservedRunningTime="2026-07-21 17:33:43.268531142 +0000 UTC m=+1903.233430745"
Jul 21 17:33:44.253707 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:44.253667    2561 generic.go:356] "Generic (PLEG): container finished" podID="48e6de2e-d73e-42dc-a316-e5f552c27d0d" containerID="65d08149213dc6f6ab08c09af0ad64bc3b39d1c3b9eeacc1dc84a088827a3f39" exitCode=0
Jul 21 17:33:44.254097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:44.253749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz" event={"ID":"48e6de2e-d73e-42dc-a316-e5f552c27d0d","Type":"ContainerDied","Data":"65d08149213dc6f6ab08c09af0ad64bc3b39d1c3b9eeacc1dc84a088827a3f39"}
Jul 21 17:33:45.386148 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:45.386119    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:45.422922 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:45.422893    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e6de2e-d73e-42dc-a316-e5f552c27d0d-maas-api-service-ca\") pod \"48e6de2e-d73e-42dc-a316-e5f552c27d0d\" (UID: \"48e6de2e-d73e-42dc-a316-e5f552c27d0d\") "
Jul 21 17:33:45.423201 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:45.423180    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e6de2e-d73e-42dc-a316-e5f552c27d0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48e6de2e-d73e-42dc-a316-e5f552c27d0d" (UID: "48e6de2e-d73e-42dc-a316-e5f552c27d0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:33:45.524185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:45.524103    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e6de2e-d73e-42dc-a316-e5f552c27d0d-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:33:46.266479 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:46.266451    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"
Jul 21 17:33:46.266479 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:46.266466    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz" event={"ID":"48e6de2e-d73e-42dc-a316-e5f552c27d0d","Type":"ContainerDied","Data":"094bac215f3c1cc2064b2fa22c760bc82d72c98c26576843f9b1f8407cd763e4"}
Jul 21 17:33:46.266734 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:46.266516    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="094bac215f3c1cc2064b2fa22c760bc82d72c98c26576843f9b1f8407cd763e4"
Jul 21 17:33:52.455938 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.455901    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"]
Jul 21 17:33:52.456392 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.456190    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q" podUID="d9e197ea-35c5-4f14-915c-37af9d6a1bed" containerName="maas-api" containerID="cri-o://47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c" gracePeriod=30
Jul 21 17:33:52.709766 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.709705    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:52.784685 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.784652    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52h29\" (UniqueName: \"kubernetes.io/projected/d9e197ea-35c5-4f14-915c-37af9d6a1bed-kube-api-access-52h29\") pod \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") "
Jul 21 17:33:52.784881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.784705    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9e197ea-35c5-4f14-915c-37af9d6a1bed-maas-api-tls\") pod \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") "
Jul 21 17:33:52.784881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.784742    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e197ea-35c5-4f14-915c-37af9d6a1bed-openshift-service-ca\") pod \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\" (UID: \"d9e197ea-35c5-4f14-915c-37af9d6a1bed\") "
Jul 21 17:33:52.785186 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.785159    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9e197ea-35c5-4f14-915c-37af9d6a1bed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9e197ea-35c5-4f14-915c-37af9d6a1bed" (UID: "d9e197ea-35c5-4f14-915c-37af9d6a1bed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:33:52.787017 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.786988    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9e197ea-35c5-4f14-915c-37af9d6a1bed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9e197ea-35c5-4f14-915c-37af9d6a1bed" (UID: "d9e197ea-35c5-4f14-915c-37af9d6a1bed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:33:52.787110 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.787043    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e197ea-35c5-4f14-915c-37af9d6a1bed-kube-api-access-52h29" (OuterVolumeSpecName: "kube-api-access-52h29") pod "d9e197ea-35c5-4f14-915c-37af9d6a1bed" (UID: "d9e197ea-35c5-4f14-915c-37af9d6a1bed"). InnerVolumeSpecName "kube-api-access-52h29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:33:52.886011 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.885983    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e197ea-35c5-4f14-915c-37af9d6a1bed-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:33:52.886011 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.886008    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52h29\" (UniqueName: \"kubernetes.io/projected/d9e197ea-35c5-4f14-915c-37af9d6a1bed-kube-api-access-52h29\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:33:52.886179 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:52.886017    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9e197ea-35c5-4f14-915c-37af9d6a1bed-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:33:53.302291 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.302255    2561 generic.go:356] "Generic (PLEG): container finished" podID="d9e197ea-35c5-4f14-915c-37af9d6a1bed" containerID="47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c" exitCode=0
Jul 21 17:33:53.302469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.302306    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"
Jul 21 17:33:53.302469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.302344    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q" event={"ID":"d9e197ea-35c5-4f14-915c-37af9d6a1bed","Type":"ContainerDied","Data":"47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c"}
Jul 21 17:33:53.302469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.302404    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q" event={"ID":"d9e197ea-35c5-4f14-915c-37af9d6a1bed","Type":"ContainerDied","Data":"128b6a324185e16613abcef7e6394891c3b1c6d3e66bb9297680b7ed311cbda2"}
Jul 21 17:33:53.302469 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.302432    2561 scope.go:117] "RemoveContainer" containerID="47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c"
Jul 21 17:33:53.312546 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.312526    2561 scope.go:117] "RemoveContainer" containerID="47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c"
Jul 21 17:33:53.312901 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:33:53.312877    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c\": container with ID starting with 47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c not found: ID does not exist" containerID="47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c"
Jul 21 17:33:53.312959 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.312912    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c"} err="failed to get container status \"47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c\": rpc error: code = NotFound desc = could not find container \"47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c\": container with ID starting with 47b1c0459bf81248b575d583d525091bb99338f951fef5083a908ffd441f443c not found: ID does not exist"
Jul 21 17:33:53.326526 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.326501    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"]
Jul 21 17:33:53.331659 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:53.331638    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d01307af-7667ff7d5d-4fr5q"]
Jul 21 17:33:54.513019 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:54.512985    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e197ea-35c5-4f14-915c-37af9d6a1bed" path="/var/lib/kubelet/pods/d9e197ea-35c5-4f14-915c-37af9d6a1bed/volumes"
Jul 21 17:33:57.107447 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:57.107388    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"]
Jul 21 17:33:57.111107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:57.111064    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01307af-bbbtz"]
Jul 21 17:33:58.513362 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:33:58.513323    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48e6de2e-d73e-42dc-a316-e5f552c27d0d" path="/var/lib/kubelet/pods/48e6de2e-d73e-42dc-a316-e5f552c27d0d/volumes"
Jul 21 17:34:30.026995 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.026958    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"]
Jul 21 17:34:30.029457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.027289    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e197ea-35c5-4f14-915c-37af9d6a1bed" containerName="maas-api"
Jul 21 17:34:30.029457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.027299    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e197ea-35c5-4f14-915c-37af9d6a1bed" containerName="maas-api"
Jul 21 17:34:30.029457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.027307    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e6de2e-d73e-42dc-a316-e5f552c27d0d" containerName="revoke-keys"
Jul 21 17:34:30.029457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.027313    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e6de2e-d73e-42dc-a316-e5f552c27d0d" containerName="revoke-keys"
Jul 21 17:34:30.029457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.027383    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e197ea-35c5-4f14-915c-37af9d6a1bed" containerName="maas-api"
Jul 21 17:34:30.029457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.027392    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e6de2e-d73e-42dc-a316-e5f552c27d0d" containerName="revoke-keys"
Jul 21 17:34:30.030381 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.030364    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.033404 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.033372    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-28b3afa2-serving-cert\""
Jul 21 17:34:30.033516 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.033484    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:34:30.034280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.034261    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:34:30.034407 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.034388    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:34:30.042077 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.042053    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"]
Jul 21 17:34:30.119516 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.119479    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jwjj\" (UniqueName: \"kubernetes.io/projected/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-kube-api-access-6jwjj\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.119714 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.119545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.119714 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.119622    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-openshift-service-ca\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.221048 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.221009    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jwjj\" (UniqueName: \"kubernetes.io/projected/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-kube-api-access-6jwjj\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.221212 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.221105    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.221212 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.221141    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-openshift-service-ca\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.221290 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:34:30.221248    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-serving-cert: secret "maas-api-e2e-mt-28b3afa2-serving-cert" not found
Jul 21 17:34:30.221329 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:34:30.221316    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls podName:7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd nodeName:}" failed. No retries permitted until 2026-07-21 17:34:30.721293923 +0000 UTC m=+1950.686193507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls") pod "maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" (UID: "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd") : secret "maas-api-e2e-mt-28b3afa2-serving-cert" not found
Jul 21 17:34:30.221850 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.221832    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-openshift-service-ca\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.233057 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.233036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jwjj\" (UniqueName: \"kubernetes.io/projected/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-kube-api-access-6jwjj\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.724700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.724665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.727100 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.727073    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls\") pod \"maas-api-e2e-mt-28b3afa2-58559594df-6mwz5\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:30.943234 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:30.943198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:31.093844 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:31.093808    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"]
Jul 21 17:34:31.097168 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:34:31.097139    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7841bfd6_0a88_4a58_a6ba_fbf2888dd2cd.slice/crio-2205e6f76c30a754be171f4d1e49cc8aeeeca6afeef4f46d13f48ad5a351307e WatchSource:0}: Error finding container 2205e6f76c30a754be171f4d1e49cc8aeeeca6afeef4f46d13f48ad5a351307e: Status 404 returned error can't find the container with id 2205e6f76c30a754be171f4d1e49cc8aeeeca6afeef4f46d13f48ad5a351307e
Jul 21 17:34:31.098619 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:31.098598    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:34:31.516025 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:31.515983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" event={"ID":"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd","Type":"ContainerStarted","Data":"2205e6f76c30a754be171f4d1e49cc8aeeeca6afeef4f46d13f48ad5a351307e"}
Jul 21 17:34:32.522891 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:32.522853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" event={"ID":"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd","Type":"ContainerStarted","Data":"640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345"}
Jul 21 17:34:32.523270 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:32.522939    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:34:32.553323 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:32.553278    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" podStartSLOduration=1.836379258 podStartE2EDuration="2.553262648s" totalImagesPullingTime="716.88339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:34:32.551272498 +0000 UTC m=+1952.516172102" watchObservedRunningTime="2026-07-21 17:34:32.553262648 +0000 UTC m=+1952.518162253"
Jul 21 17:34:38.533995 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:34:38.533964    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:35:12.810212 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.810179    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"]
Jul 21 17:35:12.813647 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.813629    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:12.815967 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.815941    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:35:12.825748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.825725    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"]
Jul 21 17:35:12.866349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.866318    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961110d9-21ea-4f14-8e02-f4649d930641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt\" (UID: \"961110d9-21ea-4f14-8e02-f4649d930641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:12.967742 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.967705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961110d9-21ea-4f14-8e02-f4649d930641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt\" (UID: \"961110d9-21ea-4f14-8e02-f4649d930641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:12.968340 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:12.968320    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961110d9-21ea-4f14-8e02-f4649d930641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt\" (UID: \"961110d9-21ea-4f14-8e02-f4649d930641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:13.124510 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:13.124426    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:13.268743 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:13.268710    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"]
Jul 21 17:35:13.269921 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:35:13.269889    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961110d9_21ea_4f14_8e02_f4649d930641.slice/crio-f377b144d48e71e7250d16fa403aebd07fa34c8e8bd31e8dc0c02d377f15a7a8 WatchSource:0}: Error finding container f377b144d48e71e7250d16fa403aebd07fa34c8e8bd31e8dc0c02d377f15a7a8: Status 404 returned error can't find the container with id f377b144d48e71e7250d16fa403aebd07fa34c8e8bd31e8dc0c02d377f15a7a8
Jul 21 17:35:13.751279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:13.751184    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt" event={"ID":"961110d9-21ea-4f14-8e02-f4649d930641","Type":"ContainerStarted","Data":"cac95e1485c9404da459ca5919bc0a49ca34bf14b1859899514feac8d89dc468"}
Jul 21 17:35:13.751279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:13.751255    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt" event={"ID":"961110d9-21ea-4f14-8e02-f4649d930641","Type":"ContainerStarted","Data":"f377b144d48e71e7250d16fa403aebd07fa34c8e8bd31e8dc0c02d377f15a7a8"}
Jul 21 17:35:13.771397 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:13.771340    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt" podStartSLOduration=1.771323036 podStartE2EDuration="1.771323036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:35:13.768488159 +0000 UTC m=+1993.733387763" watchObservedRunningTime="2026-07-21 17:35:13.771323036 +0000 UTC m=+1993.736222646"
Jul 21 17:35:14.759465 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:14.759427    2561 generic.go:356] "Generic (PLEG): container finished" podID="961110d9-21ea-4f14-8e02-f4649d930641" containerID="cac95e1485c9404da459ca5919bc0a49ca34bf14b1859899514feac8d89dc468" exitCode=0
Jul 21 17:35:14.759907 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:14.759521    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt" event={"ID":"961110d9-21ea-4f14-8e02-f4649d930641","Type":"ContainerDied","Data":"cac95e1485c9404da459ca5919bc0a49ca34bf14b1859899514feac8d89dc468"}
Jul 21 17:35:15.899190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:15.899164    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:15.993692 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:15.993650    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961110d9-21ea-4f14-8e02-f4649d930641-maas-api-service-ca\") pod \"961110d9-21ea-4f14-8e02-f4649d930641\" (UID: \"961110d9-21ea-4f14-8e02-f4649d930641\") "
Jul 21 17:35:15.993967 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:15.993944    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/961110d9-21ea-4f14-8e02-f4649d930641-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "961110d9-21ea-4f14-8e02-f4649d930641" (UID: "961110d9-21ea-4f14-8e02-f4649d930641"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:35:16.095235 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:16.095151    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961110d9-21ea-4f14-8e02-f4649d930641-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:35:16.772351 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:16.772254    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"
Jul 21 17:35:16.772351 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:16.772254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt" event={"ID":"961110d9-21ea-4f14-8e02-f4649d930641","Type":"ContainerDied","Data":"f377b144d48e71e7250d16fa403aebd07fa34c8e8bd31e8dc0c02d377f15a7a8"}
Jul 21 17:35:16.772548 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:16.772389    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f377b144d48e71e7250d16fa403aebd07fa34c8e8bd31e8dc0c02d377f15a7a8"
Jul 21 17:35:28.410609 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.410517    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"]
Jul 21 17:35:28.411113 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.410763    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" podUID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" containerName="maas-api" containerID="cri-o://640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345" gracePeriod=30
Jul 21 17:35:28.529978 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.529936    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" podUID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.62:8443/health\": dial tcp 10.133.0.62:8443: connect: connection refused"
Jul 21 17:35:28.665230 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.665174    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:35:28.804997 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.804961    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jwjj\" (UniqueName: \"kubernetes.io/projected/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-kube-api-access-6jwjj\") pod \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") "
Jul 21 17:35:28.804997 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.804998    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-openshift-service-ca\") pod \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") "
Jul 21 17:35:28.805244 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.805075    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls\") pod \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\" (UID: \"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd\") "
Jul 21 17:35:28.805445 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.805416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" (UID: "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:35:28.807233 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.807207    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-kube-api-access-6jwjj" (OuterVolumeSpecName: "kube-api-access-6jwjj") pod "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" (UID: "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd"). InnerVolumeSpecName "kube-api-access-6jwjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:35:28.807305 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.807264    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" (UID: "7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:35:28.847091 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.847059    2561 generic.go:356] "Generic (PLEG): container finished" podID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" containerID="640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345" exitCode=0
Jul 21 17:35:28.847193 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.847105    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"
Jul 21 17:35:28.847193 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.847136    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" event={"ID":"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd","Type":"ContainerDied","Data":"640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345"}
Jul 21 17:35:28.847269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.847189    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5" event={"ID":"7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd","Type":"ContainerDied","Data":"2205e6f76c30a754be171f4d1e49cc8aeeeca6afeef4f46d13f48ad5a351307e"}
Jul 21 17:35:28.847269 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.847222    2561 scope.go:117] "RemoveContainer" containerID="640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345"
Jul 21 17:35:28.857603 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.857584    2561 scope.go:117] "RemoveContainer" containerID="640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345"
Jul 21 17:35:28.857868 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:35:28.857851    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345\": container with ID starting with 640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345 not found: ID does not exist" containerID="640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345"
Jul 21 17:35:28.857921 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.857876    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345"} err="failed to get container status \"640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345\": rpc error: code = NotFound desc = could not find container \"640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345\": container with ID starting with 640d9e468448df3fa96635af00061023efe028ae8001ccd584dcfdd7a41a5345 not found: ID does not exist"
Jul 21 17:35:28.873056 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.873031    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"]
Jul 21 17:35:28.880410 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.880384    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28b3afa2-58559594df-6mwz5"]
Jul 21 17:35:28.906766 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.906744    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jwjj\" (UniqueName: \"kubernetes.io/projected/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-kube-api-access-6jwjj\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:35:28.906766 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.906765    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:35:28.906880 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:28.906775    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:35:30.514668 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:30.514640    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" path="/var/lib/kubelet/pods/7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd/volumes"
Jul 21 17:35:32.917723 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:32.917685    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"]
Jul 21 17:35:32.920641 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:32.920615    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-28b3afa2-gl7lt"]
Jul 21 17:35:34.513512 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:35:34.513477    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="961110d9-21ea-4f14-8e02-f4649d930641" path="/var/lib/kubelet/pods/961110d9-21ea-4f14-8e02-f4649d930641/volumes"
Jul 21 17:36:01.142525 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:36:01.142487    2561 scope.go:117] "RemoveContainer" containerID="8255d5fe937d0cc3844fc9a2460606e98fe6bf6a222d13d062efbb5a6a445171"
Jul 21 17:36:01.150956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:36:01.150938    2561 scope.go:117] "RemoveContainer" containerID="a79fd0370872a4ec8292d7bd3fbb9154ab7da93829a16559a1780357182e3c83"
Jul 21 17:36:01.158943 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:36:01.158923    2561 scope.go:117] "RemoveContainer" containerID="233e13f799e52041e8f43418612896f10cc64b30f133ed3a261808673b41a635"
Jul 21 17:37:00.634964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:00.634938    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:37:00.639774 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:00.639754    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:37:01.190323 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:01.190287    2561 scope.go:117] "RemoveContainer" containerID="84ee8a47fde8bea4c4c5d58bd9d9f5da2b693bda0acffc3306b7040f072a7c17"
Jul 21 17:37:46.041115 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041073    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"]
Jul 21 17:37:46.041626 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041437    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" containerName="maas-api"
Jul 21 17:37:46.041626 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041448    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" containerName="maas-api"
Jul 21 17:37:46.041626 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041456    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961110d9-21ea-4f14-8e02-f4649d930641" containerName="revoke-keys"
Jul 21 17:37:46.041626 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041462    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="961110d9-21ea-4f14-8e02-f4649d930641" containerName="revoke-keys"
Jul 21 17:37:46.041626 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041514    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7841bfd6-0a88-4a58-a6ba-fbf2888dd2cd" containerName="maas-api"
Jul 21 17:37:46.041626 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.041526    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="961110d9-21ea-4f14-8e02-f4649d930641" containerName="revoke-keys"
Jul 21 17:37:46.044597 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.044578    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.047638 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.047622    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:37:46.048491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.048469    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2d0798-serving-cert\""
Jul 21 17:37:46.048746 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.048729    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:37:46.048840 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.048821    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:37:46.063310 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.063281    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"]
Jul 21 17:37:46.176727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.176689    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.176928 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.176741    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.176928 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.176785    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp7lb\" (UniqueName: \"kubernetes.io/projected/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-kube-api-access-vp7lb\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.278140 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.278098    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp7lb\" (UniqueName: \"kubernetes.io/projected/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-kube-api-access-vp7lb\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.278353 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.278202    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.278353 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.278227    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.278881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.278859    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.281071 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.281046    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.298131 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.298067    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp7lb\" (UniqueName: \"kubernetes.io/projected/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-kube-api-access-vp7lb\") pod \"maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.355060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.355023    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:46.497249 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.497224    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"]
Jul 21 17:37:46.499947 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:37:46.499917    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33a671c_0ad3_4c42_8d61_5074e3d4e68f.slice/crio-6cd21656c55427f4eeb3259ff6568e5a2b4416c9aff1983320cf810b3134c4a3 WatchSource:0}: Error finding container 6cd21656c55427f4eeb3259ff6568e5a2b4416c9aff1983320cf810b3134c4a3: Status 404 returned error can't find the container with id 6cd21656c55427f4eeb3259ff6568e5a2b4416c9aff1983320cf810b3134c4a3
Jul 21 17:37:46.615233 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:46.615155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst" event={"ID":"f33a671c-0ad3-4c42-8d61-5074e3d4e68f","Type":"ContainerStarted","Data":"6cd21656c55427f4eeb3259ff6568e5a2b4416c9aff1983320cf810b3134c4a3"}
Jul 21 17:37:47.621646 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:47.621607    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst" event={"ID":"f33a671c-0ad3-4c42-8d61-5074e3d4e68f","Type":"ContainerStarted","Data":"6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d"}
Jul 21 17:37:47.622097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:47.621668    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:47.642895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:47.642834    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst" podStartSLOduration=1.038350208 podStartE2EDuration="1.642817662s" totalImagesPullingTime="604.467454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:37:47.640869311 +0000 UTC m=+2147.605768914" watchObservedRunningTime="2026-07-21 17:37:47.642817662 +0000 UTC m=+2147.607717266"
Jul 21 17:37:53.632335 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:53.632261    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:37:54.100476 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.100439    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"]
Jul 21 17:37:54.104721 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.104693    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.107611 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.107579    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-892f8e-serving-cert\""
Jul 21 17:37:54.120904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.120871    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"]
Jul 21 17:37:54.252914 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.252878    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81005ca1-50c1-4dfc-823f-e57fd3a975de-openshift-service-ca\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.252914 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.252916    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h8nl\" (UniqueName: \"kubernetes.io/projected/81005ca1-50c1-4dfc-823f-e57fd3a975de-kube-api-access-4h8nl\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.253107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.252941    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81005ca1-50c1-4dfc-823f-e57fd3a975de-maas-api-tls\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.353808 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.353722    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81005ca1-50c1-4dfc-823f-e57fd3a975de-openshift-service-ca\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.353808 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.353765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h8nl\" (UniqueName: \"kubernetes.io/projected/81005ca1-50c1-4dfc-823f-e57fd3a975de-kube-api-access-4h8nl\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.353808 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.353799    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81005ca1-50c1-4dfc-823f-e57fd3a975de-maas-api-tls\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.354407 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.354390    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81005ca1-50c1-4dfc-823f-e57fd3a975de-openshift-service-ca\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.356495 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.356478    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81005ca1-50c1-4dfc-823f-e57fd3a975de-maas-api-tls\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.365377 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.365354    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h8nl\" (UniqueName: \"kubernetes.io/projected/81005ca1-50c1-4dfc-823f-e57fd3a975de-kube-api-access-4h8nl\") pod \"maas-api-e2e-inf-b-892f8e-597945767f-r85hk\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.417300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.417267    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:37:54.580917 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.580887    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"]
Jul 21 17:37:54.583064 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:37:54.583037    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81005ca1_50c1_4dfc_823f_e57fd3a975de.slice/crio-eba34e998b54bc023cb8524a483a938e789c8760793787d0065e759ed6f6116c WatchSource:0}: Error finding container eba34e998b54bc023cb8524a483a938e789c8760793787d0065e759ed6f6116c: Status 404 returned error can't find the container with id eba34e998b54bc023cb8524a483a938e789c8760793787d0065e759ed6f6116c
Jul 21 17:37:54.659746 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:54.659715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk" event={"ID":"81005ca1-50c1-4dfc-823f-e57fd3a975de","Type":"ContainerStarted","Data":"eba34e998b54bc023cb8524a483a938e789c8760793787d0065e759ed6f6116c"}
Jul 21 17:37:55.673352 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:55.673310    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk" event={"ID":"81005ca1-50c1-4dfc-823f-e57fd3a975de","Type":"ContainerStarted","Data":"8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154"}
Jul 21 17:37:55.673772 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:37:55.673412    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:38:01.211166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:01.211130    2561 scope.go:117] "RemoveContainer" containerID="23747ac04141a6260a5f783fae562d5088a61a2183d1c4d654b20cc45ad80367"
Jul 21 17:38:01.684021 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:01.683994    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:38:01.708471 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:01.708416    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk" podStartSLOduration=7.26171154 podStartE2EDuration="7.708402667s" totalImagesPullingTime="446.691127ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:37:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:37:55.705532662 +0000 UTC m=+2155.670432266" watchObservedRunningTime="2026-07-21 17:38:01.708402667 +0000 UTC m=+2161.673302272"
Jul 21 17:38:40.175672 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.175635    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"]
Jul 21 17:38:40.179007 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.178987    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.185889 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.185647    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-892f8e\"/\"test-model-892f8e-kserve-self-signed-certs\""
Jul 21 17:38:40.185889 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.185668    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-892f8e\"/\"default-dockercfg-s79vf\""
Jul 21 17:38:40.185889 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.185685    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-892f8e\"/\"kube-root-ca.crt\""
Jul 21 17:38:40.186055 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.185905    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-892f8e\"/\"openshift-service-ca.crt\""
Jul 21 17:38:40.200190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.200167    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"]
Jul 21 17:38:40.238975 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.238951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4fb4\" (UniqueName: \"kubernetes.io/projected/ea23a8cd-4141-4d35-b589-b07f838961ee-kube-api-access-k4fb4\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.239134 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.238989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-home\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.239134 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.239022    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea23a8cd-4141-4d35-b589-b07f838961ee-tls-certs\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.239134 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.239109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-dshm\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.239309 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.239138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-tmp-dir\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.239309 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.239196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-model-cache\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340004 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.339965    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-model-cache\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340072    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4fb4\" (UniqueName: \"kubernetes.io/projected/ea23a8cd-4141-4d35-b589-b07f838961ee-kube-api-access-k4fb4\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-home\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340135    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea23a8cd-4141-4d35-b589-b07f838961ee-tls-certs\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-dshm\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340189    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-tmp-dir\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340457 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340347    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-model-cache\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340518 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340466    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-home\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.340609 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.340591    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-tmp-dir\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.342576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.342524    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-dshm\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.342805 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.342787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea23a8cd-4141-4d35-b589-b07f838961ee-tls-certs\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.350751 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.350705    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4fb4\" (UniqueName: \"kubernetes.io/projected/ea23a8cd-4141-4d35-b589-b07f838961ee-kube-api-access-k4fb4\") pod \"test-model-892f8e-kserve-788b96f8bf-9dsqn\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") " pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.488982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.488899    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:40.627895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.627871    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"]
Jul 21 17:38:40.629825 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:38:40.629798    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea23a8cd_4141_4d35_b589_b07f838961ee.slice/crio-7d75ca2121ab06d26456f2289ac75b1851092328e21882eb9a25480f4fa952e4 WatchSource:0}: Error finding container 7d75ca2121ab06d26456f2289ac75b1851092328e21882eb9a25480f4fa952e4: Status 404 returned error can't find the container with id 7d75ca2121ab06d26456f2289ac75b1851092328e21882eb9a25480f4fa952e4
Jul 21 17:38:40.908744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.908701    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn" event={"ID":"ea23a8cd-4141-4d35-b589-b07f838961ee","Type":"ContainerStarted","Data":"d236140097a3520cea98f57841224a98288846686b40bfededd1a8c692e6ecda"}
Jul 21 17:38:40.908744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.908745    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn" event={"ID":"ea23a8cd-4141-4d35-b589-b07f838961ee","Type":"ContainerStarted","Data":"7d75ca2121ab06d26456f2289ac75b1851092328e21882eb9a25480f4fa952e4"}
Jul 21 17:38:40.932437 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:40.932391    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn" podStartSLOduration=0.932376464 podStartE2EDuration="932.376464ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:38:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:38:40.931702327 +0000 UTC m=+2200.896601932" watchObservedRunningTime="2026-07-21 17:38:40.932376464 +0000 UTC m=+2200.897276068"
Jul 21 17:38:50.489820 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:50.489784    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:50.490320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:50.489837    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:50.502519 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:50.502495    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:38:50.976238 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:38:50.976206    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:39:19.069849 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.069816    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"]
Jul 21 17:39:19.206899 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.206867    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"]
Jul 21 17:39:19.210494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.210474    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:19.213419 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.213401    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:39:19.219448 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.219426    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"]
Jul 21 17:39:19.291803 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.291776    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq\" (UID: \"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:19.392695 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.392670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq\" (UID: \"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:19.393250 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.393231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq\" (UID: \"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:19.522655 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.522628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:19.663085 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:19.663056    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"]
Jul 21 17:39:19.663931 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:39:19.663910    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b5e1d7_da7e_4536_9d9c_26dc4749fdc8.slice/crio-44832b53f0584a4772adf9f5dd480091e7936f49b129e6bf97ad96d9e98f3d25 WatchSource:0}: Error finding container 44832b53f0584a4772adf9f5dd480091e7936f49b129e6bf97ad96d9e98f3d25: Status 404 returned error can't find the container with id 44832b53f0584a4772adf9f5dd480091e7936f49b129e6bf97ad96d9e98f3d25
Jul 21 17:39:20.131429 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:20.131339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq" event={"ID":"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8","Type":"ContainerStarted","Data":"1226c1a3fa33181b0658f7410cd7a4bc1e4beb6c8806645899a06a00d491e520"}
Jul 21 17:39:20.131429 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:20.131414    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq" event={"ID":"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8","Type":"ContainerStarted","Data":"44832b53f0584a4772adf9f5dd480091e7936f49b129e6bf97ad96d9e98f3d25"}
Jul 21 17:39:20.153900 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:20.153843    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq" podStartSLOduration=1.1538287409999999 podStartE2EDuration="1.153828741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:39:20.151765546 +0000 UTC m=+2240.116665151" watchObservedRunningTime="2026-07-21 17:39:20.153828741 +0000 UTC m=+2240.118728360"
Jul 21 17:39:21.138143 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:21.138109    2561 generic.go:356] "Generic (PLEG): container finished" podID="17b5e1d7-da7e-4536-9d9c-26dc4749fdc8" containerID="1226c1a3fa33181b0658f7410cd7a4bc1e4beb6c8806645899a06a00d491e520" exitCode=0
Jul 21 17:39:21.138533 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:21.138190    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq" event={"ID":"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8","Type":"ContainerDied","Data":"1226c1a3fa33181b0658f7410cd7a4bc1e4beb6c8806645899a06a00d491e520"}
Jul 21 17:39:22.275878 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:22.275855    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:22.321869 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:22.321837    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8-maas-api-service-ca\") pod \"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8\" (UID: \"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8\") "
Jul 21 17:39:22.322194 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:22.322174    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17b5e1d7-da7e-4536-9d9c-26dc4749fdc8" (UID: "17b5e1d7-da7e-4536-9d9c-26dc4749fdc8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:39:22.423133 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:22.423056    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:23.150329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:23.150302    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"
Jul 21 17:39:23.150493 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:23.150300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq" event={"ID":"17b5e1d7-da7e-4536-9d9c-26dc4749fdc8","Type":"ContainerDied","Data":"44832b53f0584a4772adf9f5dd480091e7936f49b129e6bf97ad96d9e98f3d25"}
Jul 21 17:39:23.150493 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:23.150433    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44832b53f0584a4772adf9f5dd480091e7936f49b129e6bf97ad96d9e98f3d25"
Jul 21 17:39:29.769964 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:29.769931    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"]
Jul 21 17:39:29.770389 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:29.770128    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst" podUID="f33a671c-0ad3-4c42-8d61-5074e3d4e68f" containerName="maas-api" containerID="cri-o://6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d" gracePeriod=30
Jul 21 17:39:30.018831 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.018806    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:39:30.088122 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.087968    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-openshift-service-ca\") pod \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") "
Jul 21 17:39:30.088122 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.088045    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp7lb\" (UniqueName: \"kubernetes.io/projected/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-kube-api-access-vp7lb\") pod \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") "
Jul 21 17:39:30.088122 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.088066    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-maas-api-tls\") pod \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\" (UID: \"f33a671c-0ad3-4c42-8d61-5074e3d4e68f\") "
Jul 21 17:39:30.096976 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.088332    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f33a671c-0ad3-4c42-8d61-5074e3d4e68f" (UID: "f33a671c-0ad3-4c42-8d61-5074e3d4e68f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:39:30.096976 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.090198    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-kube-api-access-vp7lb" (OuterVolumeSpecName: "kube-api-access-vp7lb") pod "f33a671c-0ad3-4c42-8d61-5074e3d4e68f" (UID: "f33a671c-0ad3-4c42-8d61-5074e3d4e68f"). InnerVolumeSpecName "kube-api-access-vp7lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:39:30.096976 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.090243    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f33a671c-0ad3-4c42-8d61-5074e3d4e68f" (UID: "f33a671c-0ad3-4c42-8d61-5074e3d4e68f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:39:30.189364 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.189334    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp7lb\" (UniqueName: \"kubernetes.io/projected/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-kube-api-access-vp7lb\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:30.189364 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.189366    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:30.189532 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.189398    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33a671c-0ad3-4c42-8d61-5074e3d4e68f-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:30.191599 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.191572    2561 generic.go:356] "Generic (PLEG): container finished" podID="f33a671c-0ad3-4c42-8d61-5074e3d4e68f" containerID="6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d" exitCode=0
Jul 21 17:39:30.191712 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.191633    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"
Jul 21 17:39:30.191712 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.191650    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst" event={"ID":"f33a671c-0ad3-4c42-8d61-5074e3d4e68f","Type":"ContainerDied","Data":"6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d"}
Jul 21 17:39:30.191827 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.191718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst" event={"ID":"f33a671c-0ad3-4c42-8d61-5074e3d4e68f","Type":"ContainerDied","Data":"6cd21656c55427f4eeb3259ff6568e5a2b4416c9aff1983320cf810b3134c4a3"}
Jul 21 17:39:30.191827 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.191751    2561 scope.go:117] "RemoveContainer" containerID="6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d"
Jul 21 17:39:30.201261 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.201243    2561 scope.go:117] "RemoveContainer" containerID="6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d"
Jul 21 17:39:30.201516 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:39:30.201499    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d\": container with ID starting with 6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d not found: ID does not exist" containerID="6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d"
Jul 21 17:39:30.201604 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.201529    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d"} err="failed to get container status \"6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d\": rpc error: code = NotFound desc = could not find container \"6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d\": container with ID starting with 6fd55e8a93ae4df393d9565e04ef7d572058089625a3c503243a1a6b5d39d27d not found: ID does not exist"
Jul 21 17:39:30.215659 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.215639    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"]
Jul 21 17:39:30.221185 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.221166    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d0798-76c9579b58-hmkst"]
Jul 21 17:39:30.513776 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:30.513743    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f33a671c-0ad3-4c42-8d61-5074e3d4e68f" path="/var/lib/kubelet/pods/f33a671c-0ad3-4c42-8d61-5074e3d4e68f/volumes"
Jul 21 17:39:34.090222 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.090177    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn" podUID="ea23a8cd-4141-4d35-b589-b07f838961ee" containerName="main" containerID="cri-o://d236140097a3520cea98f57841224a98288846686b40bfededd1a8c692e6ecda" gracePeriod=45
Jul 21 17:39:34.215002 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.214972    2561 generic.go:356] "Generic (PLEG): container finished" podID="ea23a8cd-4141-4d35-b589-b07f838961ee" containerID="d236140097a3520cea98f57841224a98288846686b40bfededd1a8c692e6ecda" exitCode=0
Jul 21 17:39:34.215143 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.215041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn" event={"ID":"ea23a8cd-4141-4d35-b589-b07f838961ee","Type":"ContainerDied","Data":"d236140097a3520cea98f57841224a98288846686b40bfededd1a8c692e6ecda"}
Jul 21 17:39:34.353601 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.353509    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:39:34.370276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.370254    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"]
Jul 21 17:39:34.373735 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.373709    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d0798-g6mkq"]
Jul 21 17:39:34.424924 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.424903    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea23a8cd-4141-4d35-b589-b07f838961ee-tls-certs\") pod \"ea23a8cd-4141-4d35-b589-b07f838961ee\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") "
Jul 21 17:39:34.425044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.424937    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-tmp-dir\") pod \"ea23a8cd-4141-4d35-b589-b07f838961ee\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") "
Jul 21 17:39:34.425044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.424956    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-home\") pod \"ea23a8cd-4141-4d35-b589-b07f838961ee\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") "
Jul 21 17:39:34.425044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.424997    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-model-cache\") pod \"ea23a8cd-4141-4d35-b589-b07f838961ee\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") "
Jul 21 17:39:34.425044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.425024    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-dshm\") pod \"ea23a8cd-4141-4d35-b589-b07f838961ee\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") "
Jul 21 17:39:34.425224 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.425095    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4fb4\" (UniqueName: \"kubernetes.io/projected/ea23a8cd-4141-4d35-b589-b07f838961ee-kube-api-access-k4fb4\") pod \"ea23a8cd-4141-4d35-b589-b07f838961ee\" (UID: \"ea23a8cd-4141-4d35-b589-b07f838961ee\") "
Jul 21 17:39:34.425281 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.425201    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea23a8cd-4141-4d35-b589-b07f838961ee" (UID: "ea23a8cd-4141-4d35-b589-b07f838961ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:39:34.425337 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.425292    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-home" (OuterVolumeSpecName: "home") pod "ea23a8cd-4141-4d35-b589-b07f838961ee" (UID: "ea23a8cd-4141-4d35-b589-b07f838961ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:39:34.425337 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.425307    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea23a8cd-4141-4d35-b589-b07f838961ee" (UID: "ea23a8cd-4141-4d35-b589-b07f838961ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:39:34.425337 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.425330    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-tmp-dir\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:34.427229 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.427205    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea23a8cd-4141-4d35-b589-b07f838961ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea23a8cd-4141-4d35-b589-b07f838961ee" (UID: "ea23a8cd-4141-4d35-b589-b07f838961ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:39:34.427336 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.427309    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea23a8cd-4141-4d35-b589-b07f838961ee-kube-api-access-k4fb4" (OuterVolumeSpecName: "kube-api-access-k4fb4") pod "ea23a8cd-4141-4d35-b589-b07f838961ee" (UID: "ea23a8cd-4141-4d35-b589-b07f838961ee"). InnerVolumeSpecName "kube-api-access-k4fb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:39:34.427387 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.427364    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-dshm" (OuterVolumeSpecName: "dshm") pod "ea23a8cd-4141-4d35-b589-b07f838961ee" (UID: "ea23a8cd-4141-4d35-b589-b07f838961ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:39:34.514076 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.514050    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17b5e1d7-da7e-4536-9d9c-26dc4749fdc8" path="/var/lib/kubelet/pods/17b5e1d7-da7e-4536-9d9c-26dc4749fdc8/volumes"
Jul 21 17:39:34.526485 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.526466    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4fb4\" (UniqueName: \"kubernetes.io/projected/ea23a8cd-4141-4d35-b589-b07f838961ee-kube-api-access-k4fb4\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:34.526485 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.526485    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea23a8cd-4141-4d35-b589-b07f838961ee-tls-certs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:34.526657 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.526496    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-home\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:34.526657 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.526504    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-model-cache\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:34.526657 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:34.526511    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea23a8cd-4141-4d35-b589-b07f838961ee-dshm\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:35.222171 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:35.222133    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"
Jul 21 17:39:35.222547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:35.222128    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn" event={"ID":"ea23a8cd-4141-4d35-b589-b07f838961ee","Type":"ContainerDied","Data":"7d75ca2121ab06d26456f2289ac75b1851092328e21882eb9a25480f4fa952e4"}
Jul 21 17:39:35.222547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:35.222222    2561 scope.go:117] "RemoveContainer" containerID="d236140097a3520cea98f57841224a98288846686b40bfededd1a8c692e6ecda"
Jul 21 17:39:35.244491 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:35.244466    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"]
Jul 21 17:39:35.253832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:35.253802    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-892f8e/test-model-892f8e-kserve-788b96f8bf-9dsqn"]
Jul 21 17:39:36.513460 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:36.513424    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea23a8cd-4141-4d35-b589-b07f838961ee" path="/var/lib/kubelet/pods/ea23a8cd-4141-4d35-b589-b07f838961ee/volumes"
Jul 21 17:39:41.961587 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.961523    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"]
Jul 21 17:39:41.962073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962041    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33a671c-0ad3-4c42-8d61-5074e3d4e68f" containerName="maas-api"
Jul 21 17:39:41.962073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962060    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33a671c-0ad3-4c42-8d61-5074e3d4e68f" containerName="maas-api"
Jul 21 17:39:41.962204 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962075    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b5e1d7-da7e-4536-9d9c-26dc4749fdc8" containerName="revoke-keys"
Jul 21 17:39:41.962204 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962084    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b5e1d7-da7e-4536-9d9c-26dc4749fdc8" containerName="revoke-keys"
Jul 21 17:39:41.962204 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962112    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea23a8cd-4141-4d35-b589-b07f838961ee" containerName="main"
Jul 21 17:39:41.962204 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962123    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea23a8cd-4141-4d35-b589-b07f838961ee" containerName="main"
Jul 21 17:39:41.962398 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962220    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea23a8cd-4141-4d35-b589-b07f838961ee" containerName="main"
Jul 21 17:39:41.962398 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962233    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="17b5e1d7-da7e-4536-9d9c-26dc4749fdc8" containerName="revoke-keys"
Jul 21 17:39:41.962398 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.962247    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f33a671c-0ad3-4c42-8d61-5074e3d4e68f" containerName="maas-api"
Jul 21 17:39:41.966353 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.966329    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:41.968721 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.968705    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:39:41.974158 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:41.974131    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"]
Jul 21 17:39:42.093607 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:42.093539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7937a41-14e2-486d-89bd-43b74cb41f32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k\" (UID: \"a7937a41-14e2-486d-89bd-43b74cb41f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:42.194376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:42.194336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7937a41-14e2-486d-89bd-43b74cb41f32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k\" (UID: \"a7937a41-14e2-486d-89bd-43b74cb41f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:42.194989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:42.194970    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7937a41-14e2-486d-89bd-43b74cb41f32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k\" (UID: \"a7937a41-14e2-486d-89bd-43b74cb41f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:42.278322 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:42.278241    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:42.425985 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:42.425951    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"]
Jul 21 17:39:42.427587 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:39:42.427542    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7937a41_14e2_486d_89bd_43b74cb41f32.slice/crio-9f3e7b5a2ea40c59249591e03fb655c965c0a51d2d62ad7df7debd8d7cd4ef81 WatchSource:0}: Error finding container 9f3e7b5a2ea40c59249591e03fb655c965c0a51d2d62ad7df7debd8d7cd4ef81: Status 404 returned error can't find the container with id 9f3e7b5a2ea40c59249591e03fb655c965c0a51d2d62ad7df7debd8d7cd4ef81
Jul 21 17:39:43.271442 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:43.271405    2561 generic.go:356] "Generic (PLEG): container finished" podID="a7937a41-14e2-486d-89bd-43b74cb41f32" containerID="928ecc808f7448bac2e261611b9497f4f56f6b789f5cba27e79dc2d53c430644" exitCode=0
Jul 21 17:39:43.271848 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:43.271507    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k" event={"ID":"a7937a41-14e2-486d-89bd-43b74cb41f32","Type":"ContainerDied","Data":"928ecc808f7448bac2e261611b9497f4f56f6b789f5cba27e79dc2d53c430644"}
Jul 21 17:39:43.271848 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:43.271622    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k" event={"ID":"a7937a41-14e2-486d-89bd-43b74cb41f32","Type":"ContainerStarted","Data":"9f3e7b5a2ea40c59249591e03fb655c965c0a51d2d62ad7df7debd8d7cd4ef81"}
Jul 21 17:39:44.406102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:44.406077    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:44.513663 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:44.513630    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7937a41-14e2-486d-89bd-43b74cb41f32-maas-api-service-ca\") pod \"a7937a41-14e2-486d-89bd-43b74cb41f32\" (UID: \"a7937a41-14e2-486d-89bd-43b74cb41f32\") "
Jul 21 17:39:44.513963 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:44.513943    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7937a41-14e2-486d-89bd-43b74cb41f32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7937a41-14e2-486d-89bd-43b74cb41f32" (UID: "a7937a41-14e2-486d-89bd-43b74cb41f32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:39:44.615221 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:44.615152    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7937a41-14e2-486d-89bd-43b74cb41f32-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:45.283754 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:45.283725    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"
Jul 21 17:39:45.283754 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:45.283747    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k" event={"ID":"a7937a41-14e2-486d-89bd-43b74cb41f32","Type":"ContainerDied","Data":"9f3e7b5a2ea40c59249591e03fb655c965c0a51d2d62ad7df7debd8d7cd4ef81"}
Jul 21 17:39:45.283956 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:45.283789    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f3e7b5a2ea40c59249591e03fb655c965c0a51d2d62ad7df7debd8d7cd4ef81"
Jul 21 17:39:52.616021 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.615977    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"]
Jul 21 17:39:52.616488 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.616202    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk" podUID="81005ca1-50c1-4dfc-823f-e57fd3a975de" containerName="maas-api" containerID="cri-o://8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154" gracePeriod=30
Jul 21 17:39:52.865228 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.865192    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:39:52.984807 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.984771    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81005ca1-50c1-4dfc-823f-e57fd3a975de-openshift-service-ca\") pod \"81005ca1-50c1-4dfc-823f-e57fd3a975de\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") "
Jul 21 17:39:52.984992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.984821    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81005ca1-50c1-4dfc-823f-e57fd3a975de-maas-api-tls\") pod \"81005ca1-50c1-4dfc-823f-e57fd3a975de\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") "
Jul 21 17:39:52.984992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.984854    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h8nl\" (UniqueName: \"kubernetes.io/projected/81005ca1-50c1-4dfc-823f-e57fd3a975de-kube-api-access-4h8nl\") pod \"81005ca1-50c1-4dfc-823f-e57fd3a975de\" (UID: \"81005ca1-50c1-4dfc-823f-e57fd3a975de\") "
Jul 21 17:39:52.985186 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.985166    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81005ca1-50c1-4dfc-823f-e57fd3a975de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81005ca1-50c1-4dfc-823f-e57fd3a975de" (UID: "81005ca1-50c1-4dfc-823f-e57fd3a975de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:39:52.987261 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.987229    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81005ca1-50c1-4dfc-823f-e57fd3a975de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81005ca1-50c1-4dfc-823f-e57fd3a975de" (UID: "81005ca1-50c1-4dfc-823f-e57fd3a975de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:39:52.987363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:52.987286    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81005ca1-50c1-4dfc-823f-e57fd3a975de-kube-api-access-4h8nl" (OuterVolumeSpecName: "kube-api-access-4h8nl") pod "81005ca1-50c1-4dfc-823f-e57fd3a975de" (UID: "81005ca1-50c1-4dfc-823f-e57fd3a975de"). InnerVolumeSpecName "kube-api-access-4h8nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:39:53.086194 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.086164    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81005ca1-50c1-4dfc-823f-e57fd3a975de-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:53.086194 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.086190    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81005ca1-50c1-4dfc-823f-e57fd3a975de-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:53.086194 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.086199    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h8nl\" (UniqueName: \"kubernetes.io/projected/81005ca1-50c1-4dfc-823f-e57fd3a975de-kube-api-access-4h8nl\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:39:53.325884 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.325792    2561 generic.go:356] "Generic (PLEG): container finished" podID="81005ca1-50c1-4dfc-823f-e57fd3a975de" containerID="8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154" exitCode=0
Jul 21 17:39:53.325884 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.325848    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"
Jul 21 17:39:53.326174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.325886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk" event={"ID":"81005ca1-50c1-4dfc-823f-e57fd3a975de","Type":"ContainerDied","Data":"8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154"}
Jul 21 17:39:53.326174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.325947    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk" event={"ID":"81005ca1-50c1-4dfc-823f-e57fd3a975de","Type":"ContainerDied","Data":"eba34e998b54bc023cb8524a483a938e789c8760793787d0065e759ed6f6116c"}
Jul 21 17:39:53.326174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.325970    2561 scope.go:117] "RemoveContainer" containerID="8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154"
Jul 21 17:39:53.335837 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.335818    2561 scope.go:117] "RemoveContainer" containerID="8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154"
Jul 21 17:39:53.336083 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:39:53.336063    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154\": container with ID starting with 8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154 not found: ID does not exist" containerID="8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154"
Jul 21 17:39:53.336136 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.336094    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154"} err="failed to get container status \"8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154\": rpc error: code = NotFound desc = could not find container \"8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154\": container with ID starting with 8d3af7191699482f3db42397d72142f802f87076e1e11823fd97190e559f4154 not found: ID does not exist"
Jul 21 17:39:53.350809 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.350780    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"]
Jul 21 17:39:53.359244 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:53.359194    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-892f8e-597945767f-r85hk"]
Jul 21 17:39:54.513920 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:54.513887    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81005ca1-50c1-4dfc-823f-e57fd3a975de" path="/var/lib/kubelet/pods/81005ca1-50c1-4dfc-823f-e57fd3a975de/volumes"
Jul 21 17:39:57.078070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:57.078033    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"]
Jul 21 17:39:57.082357 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:57.082331    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-892f8e-5vc9k"]
Jul 21 17:39:58.513860 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:39:58.513830    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7937a41-14e2-486d-89bd-43b74cb41f32" path="/var/lib/kubelet/pods/a7937a41-14e2-486d-89bd-43b74cb41f32/volumes"
Jul 21 17:40:01.241131 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:01.241097    2561 scope.go:117] "RemoveContainer" containerID="65d08149213dc6f6ab08c09af0ad64bc3b39d1c3b9eeacc1dc84a088827a3f39"
Jul 21 17:40:08.827256 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827221    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"]
Jul 21 17:40:08.827744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827601    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81005ca1-50c1-4dfc-823f-e57fd3a975de" containerName="maas-api"
Jul 21 17:40:08.827744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827614    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="81005ca1-50c1-4dfc-823f-e57fd3a975de" containerName="maas-api"
Jul 21 17:40:08.827744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827636    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7937a41-14e2-486d-89bd-43b74cb41f32" containerName="revoke-keys"
Jul 21 17:40:08.827744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827642    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7937a41-14e2-486d-89bd-43b74cb41f32" containerName="revoke-keys"
Jul 21 17:40:08.827744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827704    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7937a41-14e2-486d-89bd-43b74cb41f32" containerName="revoke-keys"
Jul 21 17:40:08.827744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.827712    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="81005ca1-50c1-4dfc-823f-e57fd3a975de" containerName="maas-api"
Jul 21 17:40:08.830739 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.830719    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:08.835377 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.835357    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:40:08.838287 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.838268    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-32299c-serving-cert\""
Jul 21 17:40:08.838430 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.838410    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:40:08.838508 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.838489    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:40:08.848065 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.848040    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"]
Jul 21 17:40:08.914335 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.914295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5be3284-7908-40ee-a55d-6ce63c83fabf-openshift-service-ca\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:08.914508 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.914351    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:08.914508 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:08.914459    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snfvq\" (UniqueName: \"kubernetes.io/projected/b5be3284-7908-40ee-a55d-6ce63c83fabf-kube-api-access-snfvq\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.014971 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.014932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snfvq\" (UniqueName: \"kubernetes.io/projected/b5be3284-7908-40ee-a55d-6ce63c83fabf-kube-api-access-snfvq\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.015166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.014988    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5be3284-7908-40ee-a55d-6ce63c83fabf-openshift-service-ca\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.015166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.015021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.015166 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:40:09.015114    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-serving-cert: secret "maas-api-e2e-api-a-32299c-serving-cert" not found
Jul 21 17:40:09.015366 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:40:09.015179    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls podName:b5be3284-7908-40ee-a55d-6ce63c83fabf nodeName:}" failed. No retries permitted until 2026-07-21 17:40:09.515160994 +0000 UTC m=+2289.480060579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls") pod "maas-api-e2e-api-a-32299c-dd897547c-bngr8" (UID: "b5be3284-7908-40ee-a55d-6ce63c83fabf") : secret "maas-api-e2e-api-a-32299c-serving-cert" not found
Jul 21 17:40:09.015750 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.015725    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5be3284-7908-40ee-a55d-6ce63c83fabf-openshift-service-ca\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.031934 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.031910    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snfvq\" (UniqueName: \"kubernetes.io/projected/b5be3284-7908-40ee-a55d-6ce63c83fabf-kube-api-access-snfvq\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.518423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.518386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.521176 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.521150    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls\") pod \"maas-api-e2e-api-a-32299c-dd897547c-bngr8\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.740973 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.740922    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:09.885128 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.885100    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"]
Jul 21 17:40:09.887372 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:40:09.887347    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5be3284_7908_40ee_a55d_6ce63c83fabf.slice/crio-be61f2cbac18d00fe087e3fa4c55d8a318560e0082b35d444ea1ea1b55609afd WatchSource:0}: Error finding container be61f2cbac18d00fe087e3fa4c55d8a318560e0082b35d444ea1ea1b55609afd: Status 404 returned error can't find the container with id be61f2cbac18d00fe087e3fa4c55d8a318560e0082b35d444ea1ea1b55609afd
Jul 21 17:40:09.889275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:09.889252    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:40:10.425293 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:10.425200    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8" event={"ID":"b5be3284-7908-40ee-a55d-6ce63c83fabf","Type":"ContainerStarted","Data":"be61f2cbac18d00fe087e3fa4c55d8a318560e0082b35d444ea1ea1b55609afd"}
Jul 21 17:40:11.432076 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:11.432030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8" event={"ID":"b5be3284-7908-40ee-a55d-6ce63c83fabf","Type":"ContainerStarted","Data":"47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3"}
Jul 21 17:40:11.432518 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:11.432139    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:11.461482 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:11.461428    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8" podStartSLOduration=2.915966572 podStartE2EDuration="3.461413043s" totalImagesPullingTime="545.446471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:40:11.46025894 +0000 UTC m=+2291.425158551" watchObservedRunningTime="2026-07-21 17:40:11.461413043 +0000 UTC m=+2291.426312697"
Jul 21 17:40:16.731700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.731667    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"]
Jul 21 17:40:16.735165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.735144    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.738583 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.738537    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-17f82c-serving-cert\""
Jul 21 17:40:16.752913 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.752887    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"]
Jul 21 17:40:16.792878 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.792847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9q79\" (UniqueName: \"kubernetes.io/projected/83648d6e-8f50-4298-8596-1b8d8826dcaf-kube-api-access-m9q79\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.793006 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.792915    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.793006 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.792956    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83648d6e-8f50-4298-8596-1b8d8826dcaf-openshift-service-ca\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.893935 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.893905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9q79\" (UniqueName: \"kubernetes.io/projected/83648d6e-8f50-4298-8596-1b8d8826dcaf-kube-api-access-m9q79\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.894079 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.893959    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.894079 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.893982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83648d6e-8f50-4298-8596-1b8d8826dcaf-openshift-service-ca\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.894207 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:40:16.894126    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-serving-cert: secret "maas-api-e2e-api-b-17f82c-serving-cert" not found
Jul 21 17:40:16.894269 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:40:16.894214    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls podName:83648d6e-8f50-4298-8596-1b8d8826dcaf nodeName:}" failed. No retries permitted until 2026-07-21 17:40:17.394192153 +0000 UTC m=+2297.359091737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls") pod "maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" (UID: "83648d6e-8f50-4298-8596-1b8d8826dcaf") : secret "maas-api-e2e-api-b-17f82c-serving-cert" not found
Jul 21 17:40:16.894628 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.894611    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83648d6e-8f50-4298-8596-1b8d8826dcaf-openshift-service-ca\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:16.913761 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:16.913731    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9q79\" (UniqueName: \"kubernetes.io/projected/83648d6e-8f50-4298-8596-1b8d8826dcaf-kube-api-access-m9q79\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:17.399619 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:17.399585    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:17.402182 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:17.402143    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls\") pod \"maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:17.443072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:17.443040    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:17.644926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:17.644894    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:17.790871 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:17.790846    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"]
Jul 21 17:40:17.793283 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:40:17.793260    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83648d6e_8f50_4298_8596_1b8d8826dcaf.slice/crio-e0f0b73980fed195acd753796807a8992da803f5bae12fc9d3764960eea0563d WatchSource:0}: Error finding container e0f0b73980fed195acd753796807a8992da803f5bae12fc9d3764960eea0563d: Status 404 returned error can't find the container with id e0f0b73980fed195acd753796807a8992da803f5bae12fc9d3764960eea0563d
Jul 21 17:40:18.472731 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:18.472637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" event={"ID":"83648d6e-8f50-4298-8596-1b8d8826dcaf","Type":"ContainerStarted","Data":"4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73"}
Jul 21 17:40:18.472731 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:18.472691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" event={"ID":"83648d6e-8f50-4298-8596-1b8d8826dcaf","Type":"ContainerStarted","Data":"e0f0b73980fed195acd753796807a8992da803f5bae12fc9d3764960eea0563d"}
Jul 21 17:40:18.472920 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:18.472759    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:18.500581 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:18.500523    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" podStartSLOduration=2.137532999 podStartE2EDuration="2.500509813s" totalImagesPullingTime="362.976814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:40:18.497850076 +0000 UTC m=+2298.462749682" watchObservedRunningTime="2026-07-21 17:40:18.500509813 +0000 UTC m=+2298.465409418"
Jul 21 17:40:24.485305 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:24.485276    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:40:29.047740 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.047708    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"]
Jul 21 17:40:29.051426 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.051404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:29.054009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.053990    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:40:29.060327 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.060301    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"]
Jul 21 17:40:29.202951 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.202920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b25e97-c280-4b76-b436-4cf41d5e5201-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-32299c-mw7bt\" (UID: \"a8b25e97-c280-4b76-b436-4cf41d5e5201\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:29.303400 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.303322    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b25e97-c280-4b76-b436-4cf41d5e5201-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-32299c-mw7bt\" (UID: \"a8b25e97-c280-4b76-b436-4cf41d5e5201\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:29.303957 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.303937    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b25e97-c280-4b76-b436-4cf41d5e5201-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-32299c-mw7bt\" (UID: \"a8b25e97-c280-4b76-b436-4cf41d5e5201\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:29.363905 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.363879    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:29.503867 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.503845    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"]
Jul 21 17:40:29.506191 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:40:29.506162    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b25e97_c280_4b76_b436_4cf41d5e5201.slice/crio-ad2cd04db8b24e01d7471183b9de1d2692465d0f93fb5d8ff25df59c2fe5418f WatchSource:0}: Error finding container ad2cd04db8b24e01d7471183b9de1d2692465d0f93fb5d8ff25df59c2fe5418f: Status 404 returned error can't find the container with id ad2cd04db8b24e01d7471183b9de1d2692465d0f93fb5d8ff25df59c2fe5418f
Jul 21 17:40:29.534364 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:29.534339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt" event={"ID":"a8b25e97-c280-4b76-b436-4cf41d5e5201","Type":"ContainerStarted","Data":"ad2cd04db8b24e01d7471183b9de1d2692465d0f93fb5d8ff25df59c2fe5418f"}
Jul 21 17:40:30.541013 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:30.540925    2561 generic.go:356] "Generic (PLEG): container finished" podID="a8b25e97-c280-4b76-b436-4cf41d5e5201" containerID="250197e791155957e16b54d7a0f8fcab51d70796baacedc7656c0e742bd2f03c" exitCode=0
Jul 21 17:40:30.541013 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:30.540969    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt" event={"ID":"a8b25e97-c280-4b76-b436-4cf41d5e5201","Type":"ContainerDied","Data":"250197e791155957e16b54d7a0f8fcab51d70796baacedc7656c0e742bd2f03c"}
Jul 21 17:40:31.681854 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:31.681828    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:31.825917 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:31.825840    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b25e97-c280-4b76-b436-4cf41d5e5201-maas-api-service-ca\") pod \"a8b25e97-c280-4b76-b436-4cf41d5e5201\" (UID: \"a8b25e97-c280-4b76-b436-4cf41d5e5201\") "
Jul 21 17:40:31.826237 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:31.826214    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b25e97-c280-4b76-b436-4cf41d5e5201-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a8b25e97-c280-4b76-b436-4cf41d5e5201" (UID: "a8b25e97-c280-4b76-b436-4cf41d5e5201"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:40:31.926817 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:31.926785    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b25e97-c280-4b76-b436-4cf41d5e5201-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:40:32.554251 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:32.554220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt" event={"ID":"a8b25e97-c280-4b76-b436-4cf41d5e5201","Type":"ContainerDied","Data":"ad2cd04db8b24e01d7471183b9de1d2692465d0f93fb5d8ff25df59c2fe5418f"}
Jul 21 17:40:32.554251 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:32.554240    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"
Jul 21 17:40:32.554480 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:32.554268    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad2cd04db8b24e01d7471183b9de1d2692465d0f93fb5d8ff25df59c2fe5418f"
Jul 21 17:40:44.950143 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:44.950109    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"]
Jul 21 17:40:44.950654 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:44.950376    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8" podUID="b5be3284-7908-40ee-a55d-6ce63c83fabf" containerName="maas-api" containerID="cri-o://47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3" gracePeriod=30
Jul 21 17:40:45.200407 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.200352    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:45.254723 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.254695    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls\") pod \"b5be3284-7908-40ee-a55d-6ce63c83fabf\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") "
Jul 21 17:40:45.254897 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.254760    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snfvq\" (UniqueName: \"kubernetes.io/projected/b5be3284-7908-40ee-a55d-6ce63c83fabf-kube-api-access-snfvq\") pod \"b5be3284-7908-40ee-a55d-6ce63c83fabf\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") "
Jul 21 17:40:45.254897 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.254788    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5be3284-7908-40ee-a55d-6ce63c83fabf-openshift-service-ca\") pod \"b5be3284-7908-40ee-a55d-6ce63c83fabf\" (UID: \"b5be3284-7908-40ee-a55d-6ce63c83fabf\") "
Jul 21 17:40:45.255279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.255249    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5be3284-7908-40ee-a55d-6ce63c83fabf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5be3284-7908-40ee-a55d-6ce63c83fabf" (UID: "b5be3284-7908-40ee-a55d-6ce63c83fabf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:40:45.257006 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.256976    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5be3284-7908-40ee-a55d-6ce63c83fabf" (UID: "b5be3284-7908-40ee-a55d-6ce63c83fabf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:40:45.257114 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.257087    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5be3284-7908-40ee-a55d-6ce63c83fabf-kube-api-access-snfvq" (OuterVolumeSpecName: "kube-api-access-snfvq") pod "b5be3284-7908-40ee-a55d-6ce63c83fabf" (UID: "b5be3284-7908-40ee-a55d-6ce63c83fabf"). InnerVolumeSpecName "kube-api-access-snfvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:40:45.355652 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.355625    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snfvq\" (UniqueName: \"kubernetes.io/projected/b5be3284-7908-40ee-a55d-6ce63c83fabf-kube-api-access-snfvq\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:40:45.355652 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.355651    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5be3284-7908-40ee-a55d-6ce63c83fabf-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:40:45.355793 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.355661    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5be3284-7908-40ee-a55d-6ce63c83fabf-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:40:45.625575 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.625477    2561 generic.go:356] "Generic (PLEG): container finished" podID="b5be3284-7908-40ee-a55d-6ce63c83fabf" containerID="47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3" exitCode=0
Jul 21 17:40:45.625575 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.625529    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"
Jul 21 17:40:45.625748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.625588    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8" event={"ID":"b5be3284-7908-40ee-a55d-6ce63c83fabf","Type":"ContainerDied","Data":"47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3"}
Jul 21 17:40:45.625748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.625650    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8" event={"ID":"b5be3284-7908-40ee-a55d-6ce63c83fabf","Type":"ContainerDied","Data":"be61f2cbac18d00fe087e3fa4c55d8a318560e0082b35d444ea1ea1b55609afd"}
Jul 21 17:40:45.625748 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.625680    2561 scope.go:117] "RemoveContainer" containerID="47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3"
Jul 21 17:40:45.635338 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.635320    2561 scope.go:117] "RemoveContainer" containerID="47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3"
Jul 21 17:40:45.635621 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:40:45.635602    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3\": container with ID starting with 47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3 not found: ID does not exist" containerID="47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3"
Jul 21 17:40:45.635680 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.635631    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3"} err="failed to get container status \"47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3\": rpc error: code = NotFound desc = could not find container \"47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3\": container with ID starting with 47ed958736579e1f37d91b07451ae4324ef0bf15c5e686c7f857cf6deef3c1e3 not found: ID does not exist"
Jul 21 17:40:45.654923 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.650508    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"]
Jul 21 17:40:45.658692 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:45.658668    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-32299c-dd897547c-bngr8"]
Jul 21 17:40:46.513227 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:46.513190    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5be3284-7908-40ee-a55d-6ce63c83fabf" path="/var/lib/kubelet/pods/b5be3284-7908-40ee-a55d-6ce63c83fabf/volumes"
Jul 21 17:40:49.150432 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:49.150391    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"]
Jul 21 17:40:49.154438 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:49.154409    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-32299c-mw7bt"]
Jul 21 17:40:50.513441 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:50.513404    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b25e97-c280-4b76-b436-4cf41d5e5201" path="/var/lib/kubelet/pods/a8b25e97-c280-4b76-b436-4cf41d5e5201/volumes"
Jul 21 17:40:56.650284 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650243    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"]
Jul 21 17:40:56.650904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650792    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b25e97-c280-4b76-b436-4cf41d5e5201" containerName="revoke-keys"
Jul 21 17:40:56.650904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650812    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b25e97-c280-4b76-b436-4cf41d5e5201" containerName="revoke-keys"
Jul 21 17:40:56.650904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650834    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5be3284-7908-40ee-a55d-6ce63c83fabf" containerName="maas-api"
Jul 21 17:40:56.650904 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650843    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5be3284-7908-40ee-a55d-6ce63c83fabf" containerName="maas-api"
Jul 21 17:40:56.651124 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650920    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5be3284-7908-40ee-a55d-6ce63c83fabf" containerName="maas-api"
Jul 21 17:40:56.651124 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.650936    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b25e97-c280-4b76-b436-4cf41d5e5201" containerName="revoke-keys"
Jul 21 17:40:56.655474 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.655451    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:40:56.657768 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.657747    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:40:56.663497 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.663477    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"]
Jul 21 17:40:56.748205 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.748166    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f266692-543a-42f0-8c0e-2018d8f398c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb\" (UID: \"5f266692-543a-42f0-8c0e-2018d8f398c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:40:56.848977 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.848937    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f266692-543a-42f0-8c0e-2018d8f398c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb\" (UID: \"5f266692-543a-42f0-8c0e-2018d8f398c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:40:56.849592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.849551    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f266692-543a-42f0-8c0e-2018d8f398c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb\" (UID: \"5f266692-543a-42f0-8c0e-2018d8f398c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:40:56.967462 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:56.967381    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:40:57.103383 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:57.103356    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"]
Jul 21 17:40:57.105849 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:40:57.105820    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f266692_543a_42f0_8c0e_2018d8f398c0.slice/crio-03571e9c1918b7c0ff4952a727d668a9f611523e4efb2d039d8150834f3c1a6a WatchSource:0}: Error finding container 03571e9c1918b7c0ff4952a727d668a9f611523e4efb2d039d8150834f3c1a6a: Status 404 returned error can't find the container with id 03571e9c1918b7c0ff4952a727d668a9f611523e4efb2d039d8150834f3c1a6a
Jul 21 17:40:57.693276 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:57.693242    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb" event={"ID":"5f266692-543a-42f0-8c0e-2018d8f398c0","Type":"ContainerStarted","Data":"0527c1104c9a8227fc64a1517755ea3cd32f304c7b57ec10ec7189340c292788"}
Jul 21 17:40:57.693696 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:57.693300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb" event={"ID":"5f266692-543a-42f0-8c0e-2018d8f398c0","Type":"ContainerStarted","Data":"03571e9c1918b7c0ff4952a727d668a9f611523e4efb2d039d8150834f3c1a6a"}
Jul 21 17:40:57.711126 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:57.711079    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb" podStartSLOduration=1.7110642870000001 podStartE2EDuration="1.711064287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:40:57.708528902 +0000 UTC m=+2337.673428507" watchObservedRunningTime="2026-07-21 17:40:57.711064287 +0000 UTC m=+2337.675963903"
Jul 21 17:40:58.700712 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:58.700675    2561 generic.go:356] "Generic (PLEG): container finished" podID="5f266692-543a-42f0-8c0e-2018d8f398c0" containerID="0527c1104c9a8227fc64a1517755ea3cd32f304c7b57ec10ec7189340c292788" exitCode=0
Jul 21 17:40:58.701174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:58.700727    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb" event={"ID":"5f266692-543a-42f0-8c0e-2018d8f398c0","Type":"ContainerDied","Data":"0527c1104c9a8227fc64a1517755ea3cd32f304c7b57ec10ec7189340c292788"}
Jul 21 17:40:59.835167 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:59.835143    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:40:59.876941 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:59.876913    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f266692-543a-42f0-8c0e-2018d8f398c0-maas-api-service-ca\") pod \"5f266692-543a-42f0-8c0e-2018d8f398c0\" (UID: \"5f266692-543a-42f0-8c0e-2018d8f398c0\") "
Jul 21 17:40:59.877354 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:59.877328    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f266692-543a-42f0-8c0e-2018d8f398c0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f266692-543a-42f0-8c0e-2018d8f398c0" (UID: "5f266692-543a-42f0-8c0e-2018d8f398c0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:40:59.978375 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:40:59.978301    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f266692-543a-42f0-8c0e-2018d8f398c0-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:41:00.715139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:00.715104    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"
Jul 21 17:41:00.715139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:00.715107    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb" event={"ID":"5f266692-543a-42f0-8c0e-2018d8f398c0","Type":"ContainerDied","Data":"03571e9c1918b7c0ff4952a727d668a9f611523e4efb2d039d8150834f3c1a6a"}
Jul 21 17:41:00.715330 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:00.715168    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03571e9c1918b7c0ff4952a727d668a9f611523e4efb2d039d8150834f3c1a6a"
Jul 21 17:41:12.160710 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.160678    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"]
Jul 21 17:41:12.161097 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.160929    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" podUID="83648d6e-8f50-4298-8596-1b8d8826dcaf" containerName="maas-api" containerID="cri-o://4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73" gracePeriod=30
Jul 21 17:41:12.409193 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.409169    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:41:12.491758 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.491686    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83648d6e-8f50-4298-8596-1b8d8826dcaf-openshift-service-ca\") pod \"83648d6e-8f50-4298-8596-1b8d8826dcaf\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") "
Jul 21 17:41:12.491887 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.491788    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9q79\" (UniqueName: \"kubernetes.io/projected/83648d6e-8f50-4298-8596-1b8d8826dcaf-kube-api-access-m9q79\") pod \"83648d6e-8f50-4298-8596-1b8d8826dcaf\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") "
Jul 21 17:41:12.491887 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.491826    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls\") pod \"83648d6e-8f50-4298-8596-1b8d8826dcaf\" (UID: \"83648d6e-8f50-4298-8596-1b8d8826dcaf\") "
Jul 21 17:41:12.492033 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.492008    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83648d6e-8f50-4298-8596-1b8d8826dcaf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83648d6e-8f50-4298-8596-1b8d8826dcaf" (UID: "83648d6e-8f50-4298-8596-1b8d8826dcaf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:41:12.493996 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.493963    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83648d6e-8f50-4298-8596-1b8d8826dcaf-kube-api-access-m9q79" (OuterVolumeSpecName: "kube-api-access-m9q79") pod "83648d6e-8f50-4298-8596-1b8d8826dcaf" (UID: "83648d6e-8f50-4298-8596-1b8d8826dcaf"). InnerVolumeSpecName "kube-api-access-m9q79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:41:12.494107 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.494018    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83648d6e-8f50-4298-8596-1b8d8826dcaf" (UID: "83648d6e-8f50-4298-8596-1b8d8826dcaf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:41:12.592413 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.592389    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9q79\" (UniqueName: \"kubernetes.io/projected/83648d6e-8f50-4298-8596-1b8d8826dcaf-kube-api-access-m9q79\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:41:12.592413 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.592411    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83648d6e-8f50-4298-8596-1b8d8826dcaf-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:41:12.592591 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.592420    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83648d6e-8f50-4298-8596-1b8d8826dcaf-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:41:12.783970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.783891    2561 generic.go:356] "Generic (PLEG): container finished" podID="83648d6e-8f50-4298-8596-1b8d8826dcaf" containerID="4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73" exitCode=0
Jul 21 17:41:12.783970 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.783949    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"
Jul 21 17:41:12.784157 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.783980    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" event={"ID":"83648d6e-8f50-4298-8596-1b8d8826dcaf","Type":"ContainerDied","Data":"4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73"}
Jul 21 17:41:12.784157 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.784037    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc" event={"ID":"83648d6e-8f50-4298-8596-1b8d8826dcaf","Type":"ContainerDied","Data":"e0f0b73980fed195acd753796807a8992da803f5bae12fc9d3764960eea0563d"}
Jul 21 17:41:12.784157 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.784059    2561 scope.go:117] "RemoveContainer" containerID="4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73"
Jul 21 17:41:12.793144 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.793129    2561 scope.go:117] "RemoveContainer" containerID="4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73"
Jul 21 17:41:12.793406 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:41:12.793385    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73\": container with ID starting with 4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73 not found: ID does not exist" containerID="4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73"
Jul 21 17:41:12.793468 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.793416    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73"} err="failed to get container status \"4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73\": rpc error: code = NotFound desc = could not find container \"4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73\": container with ID starting with 4bd6d9c26ee58bcc6c1c75983bf58cf6c3ac99e38d7758627182c3ec7e0f7a73 not found: ID does not exist"
Jul 21 17:41:12.802911 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.802891    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"]
Jul 21 17:41:12.807376 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:12.807356    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-17f82c-687f559fb7-n4vtc"]
Jul 21 17:41:14.514939 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:14.514900    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83648d6e-8f50-4298-8596-1b8d8826dcaf" path="/var/lib/kubelet/pods/83648d6e-8f50-4298-8596-1b8d8826dcaf/volumes"
Jul 21 17:41:16.744165 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:16.744122    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"]
Jul 21 17:41:16.747303 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:16.747275    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-17f82c-v5jxb"]
Jul 21 17:41:18.512977 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:18.512940    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f266692-543a-42f0-8c0e-2018d8f398c0" path="/var/lib/kubelet/pods/5f266692-543a-42f0-8c0e-2018d8f398c0/volumes"
Jul 21 17:41:28.070831 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.070794    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"]
Jul 21 17:41:28.071279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.071135    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83648d6e-8f50-4298-8596-1b8d8826dcaf" containerName="maas-api"
Jul 21 17:41:28.071279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.071146    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83648d6e-8f50-4298-8596-1b8d8826dcaf" containerName="maas-api"
Jul 21 17:41:28.071279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.071167    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f266692-543a-42f0-8c0e-2018d8f398c0" containerName="revoke-keys"
Jul 21 17:41:28.071279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.071172    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f266692-543a-42f0-8c0e-2018d8f398c0" containerName="revoke-keys"
Jul 21 17:41:28.071279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.071227    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f266692-543a-42f0-8c0e-2018d8f398c0" containerName="revoke-keys"
Jul 21 17:41:28.071279 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.071235    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="83648d6e-8f50-4298-8596-1b8d8826dcaf" containerName="maas-api"
Jul 21 17:41:28.075446 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.075414    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.078244 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.078220    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-6fedc9-serving-cert\""
Jul 21 17:41:28.078368 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.078227    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 17:41:28.078368 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.078232    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tph4c\""
Jul 21 17:41:28.078982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.078967    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 17:41:28.084698 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.084678    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"]
Jul 21 17:41:28.231202 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.231165    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvcpx\" (UniqueName: \"kubernetes.io/projected/e54e8fd1-f808-4b8f-b568-c9f831d87a39-kube-api-access-pvcpx\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.231362 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.231226    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.231362 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.231283    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54e8fd1-f808-4b8f-b568-c9f831d87a39-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.332576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.332477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.332576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.332543    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54e8fd1-f808-4b8f-b568-c9f831d87a39-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.332754 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.332607    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvcpx\" (UniqueName: \"kubernetes.io/projected/e54e8fd1-f808-4b8f-b568-c9f831d87a39-kube-api-access-pvcpx\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.332754 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:41:28.332653    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-serving-cert: secret "maas-api-e2e-shared-a-6fedc9-serving-cert" not found
Jul 21 17:41:28.332754 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:41:28.332734    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls podName:e54e8fd1-f808-4b8f-b568-c9f831d87a39 nodeName:}" failed. No retries permitted until 2026-07-21 17:41:28.832716128 +0000 UTC m=+2368.797615710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls") pod "maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" (UID: "e54e8fd1-f808-4b8f-b568-c9f831d87a39") : secret "maas-api-e2e-shared-a-6fedc9-serving-cert" not found
Jul 21 17:41:28.333229 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.333211    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54e8fd1-f808-4b8f-b568-c9f831d87a39-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.342406 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.342380    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvcpx\" (UniqueName: \"kubernetes.io/projected/e54e8fd1-f808-4b8f-b568-c9f831d87a39-kube-api-access-pvcpx\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.836833 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.836805    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.839320 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.839296    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls\") pod \"maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:28.987382 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:28.987343    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:29.128256 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:29.128228    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"]
Jul 21 17:41:29.130316 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:41:29.130292    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54e8fd1_f808_4b8f_b568_c9f831d87a39.slice/crio-44bdb76633c7844c7efb0beb09f9de0c294d12ac2fd878ac3b256ac4a44abc7e WatchSource:0}: Error finding container 44bdb76633c7844c7efb0beb09f9de0c294d12ac2fd878ac3b256ac4a44abc7e: Status 404 returned error can't find the container with id 44bdb76633c7844c7efb0beb09f9de0c294d12ac2fd878ac3b256ac4a44abc7e
Jul 21 17:41:29.887014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:29.886925    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" event={"ID":"e54e8fd1-f808-4b8f-b568-c9f831d87a39","Type":"ContainerStarted","Data":"e23161d4e180d2b7d0284965844f04819123fd0ce9bac533b23bd625d039df12"}
Jul 21 17:41:29.887014 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:29.886977    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" event={"ID":"e54e8fd1-f808-4b8f-b568-c9f831d87a39","Type":"ContainerStarted","Data":"44bdb76633c7844c7efb0beb09f9de0c294d12ac2fd878ac3b256ac4a44abc7e"}
Jul 21 17:41:29.887213 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:29.887055    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:29.908909 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:29.908857    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" podStartSLOduration=1.4408725150000001 podStartE2EDuration="1.908842757s" totalImagesPullingTime="467.970242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:41:29.907316833 +0000 UTC m=+2369.872216437" watchObservedRunningTime="2026-07-21 17:41:29.908842757 +0000 UTC m=+2369.873742361"
Jul 21 17:41:35.897853 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:35.897819    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:41:36.120006 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.119972    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"]
Jul 21 17:41:36.123988 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.123964    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.126796 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.126768    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-5a36a6-serving-cert\""
Jul 21 17:41:36.135490 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.135459    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"]
Jul 21 17:41:36.202026 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.201937    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.202026 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.202007    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8tbx\" (UniqueName: \"kubernetes.io/projected/e4316a9c-86e6-453b-b082-57db1e4e1bc9-kube-api-access-z8tbx\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.202246 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.202057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4316a9c-86e6-453b-b082-57db1e4e1bc9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.302609 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.302550    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.302778 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.302659    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8tbx\" (UniqueName: \"kubernetes.io/projected/e4316a9c-86e6-453b-b082-57db1e4e1bc9-kube-api-access-z8tbx\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.302778 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:41:36.302695    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-serving-cert: secret "maas-api-e2e-shared-b-5a36a6-serving-cert" not found
Jul 21 17:41:36.302778 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:41:36.302769    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls podName:e4316a9c-86e6-453b-b082-57db1e4e1bc9 nodeName:}" failed. No retries permitted until 2026-07-21 17:41:36.802750884 +0000 UTC m=+2376.767650467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls") pod "maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" (UID: "e4316a9c-86e6-453b-b082-57db1e4e1bc9") : secret "maas-api-e2e-shared-b-5a36a6-serving-cert" not found
Jul 21 17:41:36.302778 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.302697    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4316a9c-86e6-453b-b082-57db1e4e1bc9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.303349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.303332    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4316a9c-86e6-453b-b082-57db1e4e1bc9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.312088 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.312063    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8tbx\" (UniqueName: \"kubernetes.io/projected/e4316a9c-86e6-453b-b082-57db1e4e1bc9-kube-api-access-z8tbx\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.806044 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.806013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:36.808618 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:36.808586    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:37.037580 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:37.037533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:37.203602 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:37.203576    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"]
Jul 21 17:41:37.205463 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:41:37.205438    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4316a9c_86e6_453b_b082_57db1e4e1bc9.slice/crio-345209bc46514078cab84353ecaad952ce90cb69177ef52a0c6b23b5709a6870 WatchSource:0}: Error finding container 345209bc46514078cab84353ecaad952ce90cb69177ef52a0c6b23b5709a6870: Status 404 returned error can't find the container with id 345209bc46514078cab84353ecaad952ce90cb69177ef52a0c6b23b5709a6870
Jul 21 17:41:37.931650 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:37.931614    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" event={"ID":"e4316a9c-86e6-453b-b082-57db1e4e1bc9","Type":"ContainerStarted","Data":"59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24"}
Jul 21 17:41:37.931806 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:37.931668    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" event={"ID":"e4316a9c-86e6-453b-b082-57db1e4e1bc9","Type":"ContainerStarted","Data":"345209bc46514078cab84353ecaad952ce90cb69177ef52a0c6b23b5709a6870"}
Jul 21 17:41:37.931806 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:37.931755    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:37.961057 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:37.961013    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" podStartSLOduration=1.6183895160000001 podStartE2EDuration="1.960998662s" totalImagesPullingTime="342.609146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:41:37.958116324 +0000 UTC m=+2377.923015927" watchObservedRunningTime="2026-07-21 17:41:37.960998662 +0000 UTC m=+2377.925898265"
Jul 21 17:41:43.943564 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:43.943534    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:41:58.436526 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.436485    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"]
Jul 21 17:41:58.440526 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.440509    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.444210 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.444185    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6fedc9\"/\"default-dockercfg-qsqgm\""
Jul 21 17:41:58.444323 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.444185    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6fedc9\"/\"kube-root-ca.crt\""
Jul 21 17:41:58.444323 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.444191    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6fedc9\"/\"auth-test-model-6fedc9-kserve-self-signed-certs\""
Jul 21 17:41:58.444323 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.444193    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6fedc9\"/\"openshift-service-ca.crt\""
Jul 21 17:41:58.448852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.448699    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"]
Jul 21 17:41:58.492919 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.492886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-model-cache\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.493072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.492954    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-tmp-dir\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.493072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.493006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n48n\" (UniqueName: \"kubernetes.io/projected/72f245d6-3630-4f1b-a6ce-daca94062f41-kube-api-access-4n48n\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.493072 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.493034    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f245d6-3630-4f1b-a6ce-daca94062f41-tls-certs\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.493187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.493099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-dshm\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.493187 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.493129    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-home\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594113    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-dshm\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594155 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-home\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-model-cache\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-tmp-dir\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594358    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n48n\" (UniqueName: \"kubernetes.io/projected/72f245d6-3630-4f1b-a6ce-daca94062f41-kube-api-access-4n48n\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594415 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594385    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f245d6-3630-4f1b-a6ce-daca94062f41-tls-certs\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594668 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-home\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594668 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-model-cache\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.594744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.594715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-tmp-dir\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.596678 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.596653    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-dshm\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.596927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.596912    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f245d6-3630-4f1b-a6ce-daca94062f41-tls-certs\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.607114 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.607088    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n48n\" (UniqueName: \"kubernetes.io/projected/72f245d6-3630-4f1b-a6ce-daca94062f41-kube-api-access-4n48n\") pod \"auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") " pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.751595 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.751482    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:41:58.888600 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:58.888575    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"]
Jul 21 17:41:58.890217 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:41:58.890188    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72f245d6_3630_4f1b_a6ce_daca94062f41.slice/crio-04a68918a39e3c36fcb6ea388e59abeb47791ca423a0a6eb0dd8a12f0d2959ac WatchSource:0}: Error finding container 04a68918a39e3c36fcb6ea388e59abeb47791ca423a0a6eb0dd8a12f0d2959ac: Status 404 returned error can't find the container with id 04a68918a39e3c36fcb6ea388e59abeb47791ca423a0a6eb0dd8a12f0d2959ac
Jul 21 17:41:59.051122 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:59.051033    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq" event={"ID":"72f245d6-3630-4f1b-a6ce-daca94062f41","Type":"ContainerStarted","Data":"c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235"}
Jul 21 17:41:59.051122 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:59.051085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq" event={"ID":"72f245d6-3630-4f1b-a6ce-daca94062f41","Type":"ContainerStarted","Data":"04a68918a39e3c36fcb6ea388e59abeb47791ca423a0a6eb0dd8a12f0d2959ac"}
Jul 21 17:41:59.078595 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:41:59.078529    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq" podStartSLOduration=1.078514395 podStartE2EDuration="1.078514395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:41:59.076525351 +0000 UTC m=+2399.041424957" watchObservedRunningTime="2026-07-21 17:41:59.078514395 +0000 UTC m=+2399.043414003"
Jul 21 17:42:00.672389 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:00.672363    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:42:00.678389 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:00.678367    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:42:01.287299 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:01.287266    2561 scope.go:117] "RemoveContainer" containerID="cac95e1485c9404da459ca5919bc0a49ca34bf14b1859899514feac8d89dc468"
Jul 21 17:42:08.752055 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:08.752017    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:42:08.752055 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:08.752064    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:42:08.764849 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:08.764819    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:42:09.120749 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:42:09.120659    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:44:38.903913 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:38.903879    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"]
Jul 21 17:44:53.924513 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:53.924470    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq" podUID="72f245d6-3630-4f1b-a6ce-daca94062f41" containerName="main" containerID="cri-o://c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235" gracePeriod=45
Jul 21 17:44:54.181129 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.181072    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:44:54.326789 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.326750    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-model-cache\") pod \"72f245d6-3630-4f1b-a6ce-daca94062f41\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") "
Jul 21 17:44:54.326992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.326827    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-home\") pod \"72f245d6-3630-4f1b-a6ce-daca94062f41\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") "
Jul 21 17:44:54.326992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.326855    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-tmp-dir\") pod \"72f245d6-3630-4f1b-a6ce-daca94062f41\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") "
Jul 21 17:44:54.326992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.326869    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-dshm\") pod \"72f245d6-3630-4f1b-a6ce-daca94062f41\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") "
Jul 21 17:44:54.326992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.326894    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n48n\" (UniqueName: \"kubernetes.io/projected/72f245d6-3630-4f1b-a6ce-daca94062f41-kube-api-access-4n48n\") pod \"72f245d6-3630-4f1b-a6ce-daca94062f41\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") "
Jul 21 17:44:54.326992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.326953    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f245d6-3630-4f1b-a6ce-daca94062f41-tls-certs\") pod \"72f245d6-3630-4f1b-a6ce-daca94062f41\" (UID: \"72f245d6-3630-4f1b-a6ce-daca94062f41\") "
Jul 21 17:44:54.327277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.327059    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-model-cache" (OuterVolumeSpecName: "model-cache") pod "72f245d6-3630-4f1b-a6ce-daca94062f41" (UID: "72f245d6-3630-4f1b-a6ce-daca94062f41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:44:54.327277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.327146    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72f245d6-3630-4f1b-a6ce-daca94062f41" (UID: "72f245d6-3630-4f1b-a6ce-daca94062f41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:44:54.327277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.327159    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-home" (OuterVolumeSpecName: "home") pod "72f245d6-3630-4f1b-a6ce-daca94062f41" (UID: "72f245d6-3630-4f1b-a6ce-daca94062f41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:44:54.327277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.327248    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-model-cache\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:44:54.327277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.327267    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-home\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:44:54.327277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.327280    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-tmp-dir\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:44:54.329319 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.329297    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f245d6-3630-4f1b-a6ce-daca94062f41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72f245d6-3630-4f1b-a6ce-daca94062f41" (UID: "72f245d6-3630-4f1b-a6ce-daca94062f41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:44:54.329629 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.329602    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f245d6-3630-4f1b-a6ce-daca94062f41-kube-api-access-4n48n" (OuterVolumeSpecName: "kube-api-access-4n48n") pod "72f245d6-3630-4f1b-a6ce-daca94062f41" (UID: "72f245d6-3630-4f1b-a6ce-daca94062f41"). InnerVolumeSpecName "kube-api-access-4n48n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:44:54.329629 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.329605    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-dshm" (OuterVolumeSpecName: "dshm") pod "72f245d6-3630-4f1b-a6ce-daca94062f41" (UID: "72f245d6-3630-4f1b-a6ce-daca94062f41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:44:54.427960 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.427931    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f245d6-3630-4f1b-a6ce-daca94062f41-dshm\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:44:54.427960 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.427955    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n48n\" (UniqueName: \"kubernetes.io/projected/72f245d6-3630-4f1b-a6ce-daca94062f41-kube-api-access-4n48n\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:44:54.427960 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:54.427966    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f245d6-3630-4f1b-a6ce-daca94062f41-tls-certs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:44:55.022794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.022762    2561 generic.go:356] "Generic (PLEG): container finished" podID="72f245d6-3630-4f1b-a6ce-daca94062f41" containerID="c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235" exitCode=0
Jul 21 17:44:55.023280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.022808    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq" event={"ID":"72f245d6-3630-4f1b-a6ce-daca94062f41","Type":"ContainerDied","Data":"c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235"}
Jul 21 17:44:55.023280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.022836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq" event={"ID":"72f245d6-3630-4f1b-a6ce-daca94062f41","Type":"ContainerDied","Data":"04a68918a39e3c36fcb6ea388e59abeb47791ca423a0a6eb0dd8a12f0d2959ac"}
Jul 21 17:44:55.023280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.022852    2561 scope.go:117] "RemoveContainer" containerID="c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235"
Jul 21 17:44:55.023280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.022811    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"
Jul 21 17:44:55.032648 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.032624    2561 scope.go:117] "RemoveContainer" containerID="c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235"
Jul 21 17:44:55.032926 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:44:55.032908    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235\": container with ID starting with c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235 not found: ID does not exist" containerID="c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235"
Jul 21 17:44:55.032988 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.032935    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235"} err="failed to get container status \"c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235\": rpc error: code = NotFound desc = could not find container \"c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235\": container with ID starting with c2216e44cd0dac3c049eaad47fd7a1a918a869405a893ddad5e090b41bd88235 not found: ID does not exist"
Jul 21 17:44:55.042849 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.042824    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"]
Jul 21 17:44:55.047977 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:55.047955    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6fedc9/auth-test-model-6fedc9-kserve-5b5556c9bc-89ngq"]
Jul 21 17:44:56.513034 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:44:56.513000    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f245d6-3630-4f1b-a6ce-daca94062f41" path="/var/lib/kubelet/pods/72f245d6-3630-4f1b-a6ce-daca94062f41/volumes"
Jul 21 17:45:00.136060 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.136025    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"]
Jul 21 17:45:00.136813 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.136446    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f245d6-3630-4f1b-a6ce-daca94062f41" containerName="main"
Jul 21 17:45:00.136813 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.136473    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f245d6-3630-4f1b-a6ce-daca94062f41" containerName="main"
Jul 21 17:45:00.136813 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.136586    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f245d6-3630-4f1b-a6ce-daca94062f41" containerName="main"
Jul 21 17:45:00.140973 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.140953    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"]
Jul 21 17:45:00.141089 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.141070    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"
Jul 21 17:45:00.143447 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.143428    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:45:00.144651 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.144633    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"]
Jul 21 17:45:00.144788 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.144770    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"
Jul 21 17:45:00.147990 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.147969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"
Jul 21 17:45:00.149862 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.149845    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"]
Jul 21 17:45:00.155695 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.155668    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"]
Jul 21 17:45:00.156277 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.156255    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"
Jul 21 17:45:00.156388 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.156276    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"]
Jul 21 17:45:00.167147 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.167126    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"
Jul 21 17:45:00.199244 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.199220    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"
Jul 21 17:45:00.330140 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.330099    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"]
Jul 21 17:45:00.331096 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:45:00.331057    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca790d98_774c_4ccd_b78b_8f2dc28b32c4.slice/crio-29080b3d0976b2d2a3f996077491a7c3d7e5e1f3a32e67508d0afcfc4cbd7ad9 WatchSource:0}: Error finding container 29080b3d0976b2d2a3f996077491a7c3d7e5e1f3a32e67508d0afcfc4cbd7ad9: Status 404 returned error can't find the container with id 29080b3d0976b2d2a3f996077491a7c3d7e5e1f3a32e67508d0afcfc4cbd7ad9
Jul 21 17:45:00.355102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.355041    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"]
Jul 21 17:45:00.356083 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:45:00.356057    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf41f01_4cc8_4577_b3e3_7d9a466e582b.slice/crio-ce6ab01aedc8ea0a74e3d10f495f7aced075a3bb830b5b4320600e9b11b7ea37 WatchSource:0}: Error finding container ce6ab01aedc8ea0a74e3d10f495f7aced075a3bb830b5b4320600e9b11b7ea37: Status 404 returned error can't find the container with id ce6ab01aedc8ea0a74e3d10f495f7aced075a3bb830b5b4320600e9b11b7ea37
Jul 21 17:45:00.377864 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:00.377842    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"]
Jul 21 17:45:00.379399 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:45:00.379363    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c39a9d_d516_40a3_8d55_2f3f2ce3576c.slice/crio-1e97b5f26b6ad80286f264fa54679aa4cd2c93b95a4df4c27abd5b03cf184696 WatchSource:0}: Error finding container 1e97b5f26b6ad80286f264fa54679aa4cd2c93b95a4df4c27abd5b03cf184696: Status 404 returned error can't find the container with id 1e97b5f26b6ad80286f264fa54679aa4cd2c93b95a4df4c27abd5b03cf184696
Jul 21 17:45:01.062756 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.062660    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn" event={"ID":"ca790d98-774c-4ccd-b78b-8f2dc28b32c4","Type":"ContainerStarted","Data":"5a423bf1e61bba7c065a5914ca8621e9869422adf933a7954b84bd7e4e4f241b"}
Jul 21 17:45:01.062756 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.062735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn" event={"ID":"ca790d98-774c-4ccd-b78b-8f2dc28b32c4","Type":"ContainerStarted","Data":"29080b3d0976b2d2a3f996077491a7c3d7e5e1f3a32e67508d0afcfc4cbd7ad9"}
Jul 21 17:45:01.064356 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.064319    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw" event={"ID":"bbf41f01-4cc8-4577-b3e3-7d9a466e582b","Type":"ContainerStarted","Data":"40df6325e83e1b5f3fe581e5077a867653b8694ebc8ddab7bee7cc4fac6c0466"}
Jul 21 17:45:01.064498 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.064390    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw" event={"ID":"bbf41f01-4cc8-4577-b3e3-7d9a466e582b","Type":"ContainerStarted","Data":"ce6ab01aedc8ea0a74e3d10f495f7aced075a3bb830b5b4320600e9b11b7ea37"}
Jul 21 17:45:01.066054 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.066027    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm" event={"ID":"a8c39a9d-d516-40a3-8d55-2f3f2ce3576c","Type":"ContainerStarted","Data":"29d17759a31356fa311718f7344f2b5e2f6df91d338f7380a9d3865cab999cd8"}
Jul 21 17:45:01.066169 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.066069    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm" event={"ID":"a8c39a9d-d516-40a3-8d55-2f3f2ce3576c","Type":"ContainerStarted","Data":"1e97b5f26b6ad80286f264fa54679aa4cd2c93b95a4df4c27abd5b03cf184696"}
Jul 21 17:45:01.078781 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.078649    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn" podStartSLOduration=1.078632033 podStartE2EDuration="1.078632033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:45:01.078001237 +0000 UTC m=+2581.042900840" watchObservedRunningTime="2026-07-21 17:45:01.078632033 +0000 UTC m=+2581.043531637"
Jul 21 17:45:01.093117 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.093074    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw" podStartSLOduration=1.093060265 podStartE2EDuration="1.093060265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:45:01.091820604 +0000 UTC m=+2581.056720208" watchObservedRunningTime="2026-07-21 17:45:01.093060265 +0000 UTC m=+2581.057959868"
Jul 21 17:45:01.107304 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:01.107259    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm" podStartSLOduration=1.107244976 podStartE2EDuration="1.107244976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:45:01.106544512 +0000 UTC m=+2581.071444117" watchObservedRunningTime="2026-07-21 17:45:01.107244976 +0000 UTC m=+2581.072144579"
Jul 21 17:45:02.072828 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:02.072789    2561 generic.go:356] "Generic (PLEG): container finished" podID="a8c39a9d-d516-40a3-8d55-2f3f2ce3576c" containerID="29d17759a31356fa311718f7344f2b5e2f6df91d338f7380a9d3865cab999cd8" exitCode=0
Jul 21 17:45:02.073280 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:02.072878    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm" event={"ID":"a8c39a9d-d516-40a3-8d55-2f3f2ce3576c","Type":"ContainerDied","Data":"29d17759a31356fa311718f7344f2b5e2f6df91d338f7380a9d3865cab999cd8"}
Jul 21 17:45:02.074458 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:02.074436    2561 generic.go:356] "Generic (PLEG): container finished" podID="ca790d98-774c-4ccd-b78b-8f2dc28b32c4" containerID="5a423bf1e61bba7c065a5914ca8621e9869422adf933a7954b84bd7e4e4f241b" exitCode=0
Jul 21 17:45:02.074597 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:02.074521    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn" event={"ID":"ca790d98-774c-4ccd-b78b-8f2dc28b32c4","Type":"ContainerDied","Data":"5a423bf1e61bba7c065a5914ca8621e9869422adf933a7954b84bd7e4e4f241b"}
Jul 21 17:45:02.076101 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:02.076078    2561 generic.go:356] "Generic (PLEG): container finished" podID="bbf41f01-4cc8-4577-b3e3-7d9a466e582b" containerID="40df6325e83e1b5f3fe581e5077a867653b8694ebc8ddab7bee7cc4fac6c0466" exitCode=0
Jul 21 17:45:02.076195 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:02.076113    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw" event={"ID":"bbf41f01-4cc8-4577-b3e3-7d9a466e582b","Type":"ContainerDied","Data":"40df6325e83e1b5f3fe581e5077a867653b8694ebc8ddab7bee7cc4fac6c0466"}
Jul 21 17:45:03.223811 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:03.223785    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"
Jul 21 17:45:03.275166 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:03.275143    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"
Jul 21 17:45:03.278402 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:03.278384    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"
Jul 21 17:45:04.088723 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.088688    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"
Jul 21 17:45:04.088931 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.088737    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn" event={"ID":"ca790d98-774c-4ccd-b78b-8f2dc28b32c4","Type":"ContainerDied","Data":"29080b3d0976b2d2a3f996077491a7c3d7e5e1f3a32e67508d0afcfc4cbd7ad9"}
Jul 21 17:45:04.088931 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.088786    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29080b3d0976b2d2a3f996077491a7c3d7e5e1f3a32e67508d0afcfc4cbd7ad9"
Jul 21 17:45:04.089968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.089946    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"
Jul 21 17:45:04.090100 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.089978    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw" event={"ID":"bbf41f01-4cc8-4577-b3e3-7d9a466e582b","Type":"ContainerDied","Data":"ce6ab01aedc8ea0a74e3d10f495f7aced075a3bb830b5b4320600e9b11b7ea37"}
Jul 21 17:45:04.090100 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.090015    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce6ab01aedc8ea0a74e3d10f495f7aced075a3bb830b5b4320600e9b11b7ea37"
Jul 21 17:45:04.091275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.091255    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm" event={"ID":"a8c39a9d-d516-40a3-8d55-2f3f2ce3576c","Type":"ContainerDied","Data":"1e97b5f26b6ad80286f264fa54679aa4cd2c93b95a4df4c27abd5b03cf184696"}
Jul 21 17:45:04.091275 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.091269    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744265-hx6lm"
Jul 21 17:45:04.091437 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:04.091293    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e97b5f26b6ad80286f264fa54679aa4cd2c93b95a4df4c27abd5b03cf184696"
Jul 21 17:45:33.636095 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636009    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"]
Jul 21 17:45:33.636681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636589    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca790d98-774c-4ccd-b78b-8f2dc28b32c4" containerName="cleanup"
Jul 21 17:45:33.636681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636609    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca790d98-774c-4ccd-b78b-8f2dc28b32c4" containerName="cleanup"
Jul 21 17:45:33.636681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636634    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf41f01-4cc8-4577-b3e3-7d9a466e582b" containerName="cleanup"
Jul 21 17:45:33.636681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636643    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf41f01-4cc8-4577-b3e3-7d9a466e582b" containerName="cleanup"
Jul 21 17:45:33.636681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636660    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c39a9d-d516-40a3-8d55-2f3f2ce3576c" containerName="cleanup"
Jul 21 17:45:33.636681 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636669    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c39a9d-d516-40a3-8d55-2f3f2ce3576c" containerName="cleanup"
Jul 21 17:45:33.637003 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636755    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf41f01-4cc8-4577-b3e3-7d9a466e582b" containerName="cleanup"
Jul 21 17:45:33.637003 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636769    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c39a9d-d516-40a3-8d55-2f3f2ce3576c" containerName="cleanup"
Jul 21 17:45:33.637003 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.636779    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca790d98-774c-4ccd-b78b-8f2dc28b32c4" containerName="cleanup"
Jul 21 17:45:33.640239 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.640218    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.642726 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.642702    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"openshift-service-ca.crt\""
Jul 21 17:45:33.643719 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.643688    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"default-dockercfg-w2h92\""
Jul 21 17:45:33.643839 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.643688    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"sub-test-model-5a36a6-kserve-self-signed-certs\""
Jul 21 17:45:33.643839 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.643746    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"kube-root-ca.crt\""
Jul 21 17:45:33.648419 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.648390    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"]
Jul 21 17:45:33.760871 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.760836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tls-certs\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.761073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.760879    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-model-cache\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.761073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.760965    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-dshm\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.761073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.760994    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tmp-dir\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.761073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.761034    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhmfj\" (UniqueName: \"kubernetes.io/projected/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-kube-api-access-dhmfj\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.761073 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.761062    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-home\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862268 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862239    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-dshm\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862268 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862272    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tmp-dir\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862506 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862300    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhmfj\" (UniqueName: \"kubernetes.io/projected/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-kube-api-access-dhmfj\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862506 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-home\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862506 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tls-certs\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862506 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-model-cache\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862745 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862694    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tmp-dir\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862745 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862738    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-home\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.862856 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.862768    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-model-cache\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.864852 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.864831    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-dshm\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.865195 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.865177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tls-certs\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.872395 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.872373    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhmfj\" (UniqueName: \"kubernetes.io/projected/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-kube-api-access-dhmfj\") pod \"sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") " pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:33.951474 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:33.951446    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:34.087195 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:34.087167    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"]
Jul 21 17:45:34.088763 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:45:34.088733    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda23b5f1b_1daa_45c1_9c3b_e3f499d866dd.slice/crio-046b94d36ac26bb93c7b7abdf7e0e3e74ad7f2b02301db90a0324eaf5e358fb8 WatchSource:0}: Error finding container 046b94d36ac26bb93c7b7abdf7e0e3e74ad7f2b02301db90a0324eaf5e358fb8: Status 404 returned error can't find the container with id 046b94d36ac26bb93c7b7abdf7e0e3e74ad7f2b02301db90a0324eaf5e358fb8
Jul 21 17:45:34.257982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:34.257900    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8" event={"ID":"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd","Type":"ContainerStarted","Data":"3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e"}
Jul 21 17:45:34.257982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:34.257944    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8" event={"ID":"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd","Type":"ContainerStarted","Data":"046b94d36ac26bb93c7b7abdf7e0e3e74ad7f2b02301db90a0324eaf5e358fb8"}
Jul 21 17:45:34.279200 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:34.279138    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8" podStartSLOduration=1.279122395 podStartE2EDuration="1.279122395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:45:34.276324122 +0000 UTC m=+2614.241223727" watchObservedRunningTime="2026-07-21 17:45:34.279122395 +0000 UTC m=+2614.244021998"
Jul 21 17:45:43.952402 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:43.952366    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:43.952913 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:43.952422    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:43.965182 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:43.965156    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:45:44.328478 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:45:44.328394    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:46:01.341939 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:01.341909    2561 scope.go:117] "RemoveContainer" containerID="928ecc808f7448bac2e261611b9497f4f56f6b789f5cba27e79dc2d53c430644"
Jul 21 17:46:01.350929 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:01.350906    2561 scope.go:117] "RemoveContainer" containerID="1226c1a3fa33181b0658f7410cd7a4bc1e4beb6c8806645899a06a00d491e520"
Jul 21 17:46:18.030382 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:18.030341    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"]
Jul 21 17:46:33.052885 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.052835    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8" podUID="a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" containerName="main" containerID="cri-o://3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e" gracePeriod=45
Jul 21 17:46:33.306319 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.306264    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:46:33.369658 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369632    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-model-cache\") pod \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") "
Jul 21 17:46:33.369835 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369685    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tmp-dir\") pod \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") "
Jul 21 17:46:33.369835 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369705    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-dshm\") pod \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") "
Jul 21 17:46:33.369835 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369761    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhmfj\" (UniqueName: \"kubernetes.io/projected/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-kube-api-access-dhmfj\") pod \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") "
Jul 21 17:46:33.369835 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369786    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-home\") pod \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") "
Jul 21 17:46:33.369835 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369824    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tls-certs\") pod \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\" (UID: \"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd\") "
Jul 21 17:46:33.370108 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369892    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" (UID: "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:46:33.370108 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.369903    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" (UID: "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:46:33.370108 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.370064    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-model-cache\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:46:33.370108 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.370075    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-home" (OuterVolumeSpecName: "home") pod "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" (UID: "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:46:33.370108 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.370083    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tmp-dir\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:46:33.372076 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.372047    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-dshm" (OuterVolumeSpecName: "dshm") pod "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" (UID: "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:46:33.372076 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.372051    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-kube-api-access-dhmfj" (OuterVolumeSpecName: "kube-api-access-dhmfj") pod "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" (UID: "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd"). InnerVolumeSpecName "kube-api-access-dhmfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:46:33.372218 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.372072    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" (UID: "a23b5f1b-1daa-45c1-9c3b-e3f499d866dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:46:33.470775 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.470747    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhmfj\" (UniqueName: \"kubernetes.io/projected/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-kube-api-access-dhmfj\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:46:33.470775 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.470772    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-home\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:46:33.470927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.470782    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-tls-certs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:46:33.470927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.470790    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd-dshm\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:46:33.597968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.597884    2561 generic.go:356] "Generic (PLEG): container finished" podID="a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" containerID="3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e" exitCode=0
Jul 21 17:46:33.597968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.597929    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8" event={"ID":"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd","Type":"ContainerDied","Data":"3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e"}
Jul 21 17:46:33.597968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.597932    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"
Jul 21 17:46:33.597968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.597957    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8" event={"ID":"a23b5f1b-1daa-45c1-9c3b-e3f499d866dd","Type":"ContainerDied","Data":"046b94d36ac26bb93c7b7abdf7e0e3e74ad7f2b02301db90a0324eaf5e358fb8"}
Jul 21 17:46:33.598343 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.597975    2561 scope.go:117] "RemoveContainer" containerID="3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e"
Jul 21 17:46:33.607571 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.607537    2561 scope.go:117] "RemoveContainer" containerID="3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e"
Jul 21 17:46:33.607849 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:46:33.607832    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e\": container with ID starting with 3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e not found: ID does not exist" containerID="3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e"
Jul 21 17:46:33.607893 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.607860    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e"} err="failed to get container status \"3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e\": rpc error: code = NotFound desc = could not find container \"3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e\": container with ID starting with 3298eb87a441ba12bd443614841768f6d97ed25a0f8f9bbcfdaee02b7a9f6d9e not found: ID does not exist"
Jul 21 17:46:33.620612 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.620589    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"]
Jul 21 17:46:33.629375 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:33.627130    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/sub-test-model-5a36a6-kserve-b8ccc5d75-8ffq8"]
Jul 21 17:46:34.513832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:34.513801    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" path="/var/lib/kubelet/pods/a23b5f1b-1daa-45c1-9c3b-e3f499d866dd/volumes"
Jul 21 17:46:41.229316 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.229272    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"]
Jul 21 17:46:41.229700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.229678    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" containerName="main"
Jul 21 17:46:41.229700 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.229690    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" containerName="main"
Jul 21 17:46:41.229774 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.229751    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a23b5f1b-1daa-45c1-9c3b-e3f499d866dd" containerName="main"
Jul 21 17:46:41.232772 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.232755    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.235794 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.235773    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"kube-root-ca.crt\""
Jul 21 17:46:41.235980 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.235772    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"openshift-service-ca.crt\""
Jul 21 17:46:41.235980 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.235825    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"rate-test-model-5a36a6-kserve-self-signed-certs\""
Jul 21 17:46:41.236140 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.235849    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5a36a6\"/\"default-dockercfg-w2h92\""
Jul 21 17:46:41.243230 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.243207    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"]
Jul 21 17:46:41.339439 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.339392    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff38e638-61ad-4a75-976a-ead25af4c819-tls-certs\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.339439 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.339444    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-home\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.339689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.339471    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-model-cache\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.339689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.339541    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-dshm\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.339689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.339628    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbc8\" (UniqueName: \"kubernetes.io/projected/ff38e638-61ad-4a75-976a-ead25af4c819-kube-api-access-swbc8\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.339689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.339649    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-tmp-dir\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.440690 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.440654    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff38e638-61ad-4a75-976a-ead25af4c819-tls-certs\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.440876 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.440705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-home\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.440876 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.440838    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-model-cache\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.441007 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.440897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-dshm\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.441007 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.440933    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbc8\" (UniqueName: \"kubernetes.io/projected/ff38e638-61ad-4a75-976a-ead25af4c819-kube-api-access-swbc8\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.441007 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.440959    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-tmp-dir\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.441169 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.441131    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-home\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.441228 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.441212    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-model-cache\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.441319 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.441302    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-tmp-dir\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.443727 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.443584    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-dshm\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.443947 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.443929    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff38e638-61ad-4a75-976a-ead25af4c819-tls-certs\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.450547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.450528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbc8\" (UniqueName: \"kubernetes.io/projected/ff38e638-61ad-4a75-976a-ead25af4c819-kube-api-access-swbc8\") pod \"rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") " pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.543819 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.543723    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:41.701278 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:41.701246    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"]
Jul 21 17:46:41.702927 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:46:41.702899    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff38e638_61ad_4a75_976a_ead25af4c819.slice/crio-df767e49e42a5ce483064b2e004ddea154cd05b99248f2d893ab6f8ec876d2f7 WatchSource:0}: Error finding container df767e49e42a5ce483064b2e004ddea154cd05b99248f2d893ab6f8ec876d2f7: Status 404 returned error can't find the container with id df767e49e42a5ce483064b2e004ddea154cd05b99248f2d893ab6f8ec876d2f7
Jul 21 17:46:42.649678 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:42.649637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx" event={"ID":"ff38e638-61ad-4a75-976a-ead25af4c819","Type":"ContainerStarted","Data":"35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48"}
Jul 21 17:46:42.649678 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:42.649682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx" event={"ID":"ff38e638-61ad-4a75-976a-ead25af4c819","Type":"ContainerStarted","Data":"df767e49e42a5ce483064b2e004ddea154cd05b99248f2d893ab6f8ec876d2f7"}
Jul 21 17:46:42.671740 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:42.671690    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx" podStartSLOduration=1.671674349 podStartE2EDuration="1.671674349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:46:42.668358342 +0000 UTC m=+2682.633257946" watchObservedRunningTime="2026-07-21 17:46:42.671674349 +0000 UTC m=+2682.636573952"
Jul 21 17:46:51.544887 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:51.544797    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:51.544887 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:51.544848    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:51.558268 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:51.558243    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:46:51.710685 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:46:51.710653    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:47:00.708912 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:00.708882    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:47:00.718300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:00.718275    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qczpg_4cb01e07-87f3-42a4-a130-4110a6245b5e/ovn-acl-logging/0.log"
Jul 21 17:47:01.376238 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:01.376206    2561 scope.go:117] "RemoveContainer" containerID="0527c1104c9a8227fc64a1517755ea3cd32f304c7b57ec10ec7189340c292788"
Jul 21 17:47:01.387157 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:01.387137    2561 scope.go:117] "RemoveContainer" containerID="250197e791155957e16b54d7a0f8fcab51d70796baacedc7656c0e742bd2f03c"
Jul 21 17:47:34.722036 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:34.721999    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"]
Jul 21 17:47:49.741880 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:49.741824    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx" podUID="ff38e638-61ad-4a75-976a-ead25af4c819" containerName="main" containerID="cri-o://35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48" gracePeriod=45
Jul 21 17:47:49.996355 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:49.996300    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:47:50.017271 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.017234    2561 generic.go:356] "Generic (PLEG): container finished" podID="ff38e638-61ad-4a75-976a-ead25af4c819" containerID="35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48" exitCode=0
Jul 21 17:47:50.017422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.017280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx" event={"ID":"ff38e638-61ad-4a75-976a-ead25af4c819","Type":"ContainerDied","Data":"35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48"}
Jul 21 17:47:50.017422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.017336    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx" event={"ID":"ff38e638-61ad-4a75-976a-ead25af4c819","Type":"ContainerDied","Data":"df767e49e42a5ce483064b2e004ddea154cd05b99248f2d893ab6f8ec876d2f7"}
Jul 21 17:47:50.017422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.017349    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"
Jul 21 17:47:50.017422 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.017362    2561 scope.go:117] "RemoveContainer" containerID="35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48"
Jul 21 17:47:50.027113 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.027095    2561 scope.go:117] "RemoveContainer" containerID="35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48"
Jul 21 17:47:50.027406 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:47:50.027379    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48\": container with ID starting with 35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48 not found: ID does not exist" containerID="35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48"
Jul 21 17:47:50.027456 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.027420    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48"} err="failed to get container status \"35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48\": rpc error: code = NotFound desc = could not find container \"35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48\": container with ID starting with 35ba4f6ad3c82a8b5177658eda6c1a6aa163c0e8aff753237cd6c955e9872f48 not found: ID does not exist"
Jul 21 17:47:50.116812 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.116786    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swbc8\" (UniqueName: \"kubernetes.io/projected/ff38e638-61ad-4a75-976a-ead25af4c819-kube-api-access-swbc8\") pod \"ff38e638-61ad-4a75-976a-ead25af4c819\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") "
Jul 21 17:47:50.116926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.116819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff38e638-61ad-4a75-976a-ead25af4c819-tls-certs\") pod \"ff38e638-61ad-4a75-976a-ead25af4c819\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") "
Jul 21 17:47:50.116926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.116857    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-tmp-dir\") pod \"ff38e638-61ad-4a75-976a-ead25af4c819\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") "
Jul 21 17:47:50.116926 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.116915    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-home\") pod \"ff38e638-61ad-4a75-976a-ead25af4c819\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") "
Jul 21 17:47:50.117052 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117014    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-dshm\") pod \"ff38e638-61ad-4a75-976a-ead25af4c819\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") "
Jul 21 17:47:50.117100 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117064    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-model-cache\") pod \"ff38e638-61ad-4a75-976a-ead25af4c819\" (UID: \"ff38e638-61ad-4a75-976a-ead25af4c819\") "
Jul 21 17:47:50.117160 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117117    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff38e638-61ad-4a75-976a-ead25af4c819" (UID: "ff38e638-61ad-4a75-976a-ead25af4c819"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:47:50.117160 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117141    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-home" (OuterVolumeSpecName: "home") pod "ff38e638-61ad-4a75-976a-ead25af4c819" (UID: "ff38e638-61ad-4a75-976a-ead25af4c819"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:47:50.117360 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117336    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff38e638-61ad-4a75-976a-ead25af4c819" (UID: "ff38e638-61ad-4a75-976a-ead25af4c819"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:47:50.117423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117368    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-home\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:50.117423 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.117389    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-tmp-dir\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:50.119049 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.119022    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff38e638-61ad-4a75-976a-ead25af4c819-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff38e638-61ad-4a75-976a-ead25af4c819" (UID: "ff38e638-61ad-4a75-976a-ead25af4c819"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:47:50.119146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.119051    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-dshm" (OuterVolumeSpecName: "dshm") pod "ff38e638-61ad-4a75-976a-ead25af4c819" (UID: "ff38e638-61ad-4a75-976a-ead25af4c819"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 17:47:50.119146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.119094    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff38e638-61ad-4a75-976a-ead25af4c819-kube-api-access-swbc8" (OuterVolumeSpecName: "kube-api-access-swbc8") pod "ff38e638-61ad-4a75-976a-ead25af4c819" (UID: "ff38e638-61ad-4a75-976a-ead25af4c819"). InnerVolumeSpecName "kube-api-access-swbc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:47:50.218265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.218220    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-dshm\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:50.218265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.218261    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff38e638-61ad-4a75-976a-ead25af4c819-model-cache\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:50.218265 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.218275    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swbc8\" (UniqueName: \"kubernetes.io/projected/ff38e638-61ad-4a75-976a-ead25af4c819-kube-api-access-swbc8\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:50.218463 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.218283    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff38e638-61ad-4a75-976a-ead25af4c819-tls-certs\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:50.341018 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.340989    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"]
Jul 21 17:47:50.345901 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.345874    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5a36a6/rate-test-model-5a36a6-kserve-84f87bf8f9-jzpfx"]
Jul 21 17:47:50.514175 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:50.514134    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff38e638-61ad-4a75-976a-ead25af4c819" path="/var/lib/kubelet/pods/ff38e638-61ad-4a75-976a-ead25af4c819/volumes"
Jul 21 17:47:51.070792 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.070757    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7639-140548771608592"]
Jul 21 17:47:51.071158 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.071105    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff38e638-61ad-4a75-976a-ead25af4c819" containerName="main"
Jul 21 17:47:51.071158 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.071116    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff38e638-61ad-4a75-976a-ead25af4c819" containerName="main"
Jul 21 17:47:51.071235 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.071182    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff38e638-61ad-4a75-976a-ead25af4c819" containerName="main"
Jul 21 17:47:51.075509 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.075488    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:51.082174 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.082141    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-140548771608592"]
Jul 21 17:47:51.085895 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.085877    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rfhgq\""
Jul 21 17:47:51.228788 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.228743    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cc7r\" (UniqueName: \"kubernetes.io/projected/b1868402-daff-42b6-a9fc-6a7166e35e8e-kube-api-access-9cc7r\") pod \"test-curl-7639-140548771608592\" (UID: \"b1868402-daff-42b6-a9fc-6a7166e35e8e\") " pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:51.329480 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.329392    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cc7r\" (UniqueName: \"kubernetes.io/projected/b1868402-daff-42b6-a9fc-6a7166e35e8e-kube-api-access-9cc7r\") pod \"test-curl-7639-140548771608592\" (UID: \"b1868402-daff-42b6-a9fc-6a7166e35e8e\") " pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:51.339340 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.339306    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cc7r\" (UniqueName: \"kubernetes.io/projected/b1868402-daff-42b6-a9fc-6a7166e35e8e-kube-api-access-9cc7r\") pod \"test-curl-7639-140548771608592\" (UID: \"b1868402-daff-42b6-a9fc-6a7166e35e8e\") " pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:51.386134 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.386099    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:51.520686 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.520659    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-140548771608592"]
Jul 21 17:47:51.522711 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:47:51.522687    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1868402_daff_42b6_a9fc_6a7166e35e8e.slice/crio-602f0af8c22253ae348f8bd9928b8f0d86ffc555bef9e1b0018cf3c420561aee WatchSource:0}: Error finding container 602f0af8c22253ae348f8bd9928b8f0d86ffc555bef9e1b0018cf3c420561aee: Status 404 returned error can't find the container with id 602f0af8c22253ae348f8bd9928b8f0d86ffc555bef9e1b0018cf3c420561aee
Jul 21 17:47:51.524039 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:51.524020    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 17:47:52.032205 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:52.032165    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-140548771608592" event={"ID":"b1868402-daff-42b6-a9fc-6a7166e35e8e","Type":"ContainerStarted","Data":"602f0af8c22253ae348f8bd9928b8f0d86ffc555bef9e1b0018cf3c420561aee"}
Jul 21 17:47:53.039494 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:53.039458    2561 generic.go:356] "Generic (PLEG): container finished" podID="b1868402-daff-42b6-a9fc-6a7166e35e8e" containerID="f75f6052fde2cda2fee3cef7eaeb72ac60b7418588bf0a608d749c361303118b" exitCode=0
Jul 21 17:47:53.039936 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:53.039542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-140548771608592" event={"ID":"b1868402-daff-42b6-a9fc-6a7166e35e8e","Type":"ContainerDied","Data":"f75f6052fde2cda2fee3cef7eaeb72ac60b7418588bf0a608d749c361303118b"}
Jul 21 17:47:54.175507 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.175479    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:54.216407 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.216375    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7639-140548771608592_b1868402-daff-42b6-a9fc-6a7166e35e8e/test-curl-7639-140548771608592/0.log"
Jul 21 17:47:54.242314 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.242280    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7639-140548771608592"]
Jul 21 17:47:54.246593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.246569    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7639-140548771608592"]
Jul 21 17:47:54.256359 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.256339    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cc7r\" (UniqueName: \"kubernetes.io/projected/b1868402-daff-42b6-a9fc-6a7166e35e8e-kube-api-access-9cc7r\") pod \"b1868402-daff-42b6-a9fc-6a7166e35e8e\" (UID: \"b1868402-daff-42b6-a9fc-6a7166e35e8e\") "
Jul 21 17:47:54.258515 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.258493    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1868402-daff-42b6-a9fc-6a7166e35e8e-kube-api-access-9cc7r" (OuterVolumeSpecName: "kube-api-access-9cc7r") pod "b1868402-daff-42b6-a9fc-6a7166e35e8e" (UID: "b1868402-daff-42b6-a9fc-6a7166e35e8e"). InnerVolumeSpecName "kube-api-access-9cc7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:47:54.357070 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.357011    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cc7r\" (UniqueName: \"kubernetes.io/projected/b1868402-daff-42b6-a9fc-6a7166e35e8e-kube-api-access-9cc7r\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:54.437952 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.437925    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"]
Jul 21 17:47:54.438338 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.438321    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1868402-daff-42b6-a9fc-6a7166e35e8e" containerName="test-curl-7639-140548771608592"
Jul 21 17:47:54.438338 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.438339    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1868402-daff-42b6-a9fc-6a7166e35e8e" containerName="test-curl-7639-140548771608592"
Jul 21 17:47:54.438479 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.438433    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1868402-daff-42b6-a9fc-6a7166e35e8e" containerName="test-curl-7639-140548771608592"
Jul 21 17:47:54.441458 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.441443    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:54.443815 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.443795    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:47:54.450725 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.450699    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"]
Jul 21 17:47:54.458365 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.458336    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4e9258-9777-4685-a622-1e7609e6c04c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw\" (UID: \"4a4e9258-9777-4685-a622-1e7609e6c04c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:54.512998 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.512970    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1868402-daff-42b6-a9fc-6a7166e35e8e" path="/var/lib/kubelet/pods/b1868402-daff-42b6-a9fc-6a7166e35e8e/volumes"
Jul 21 17:47:54.559693 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.559670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4e9258-9777-4685-a622-1e7609e6c04c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw\" (UID: \"4a4e9258-9777-4685-a622-1e7609e6c04c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:54.560287 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.560270    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4e9258-9777-4685-a622-1e7609e6c04c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw\" (UID: \"4a4e9258-9777-4685-a622-1e7609e6c04c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:54.753617 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.753582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:54.890190 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:54.890154    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"]
Jul 21 17:47:54.891414 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:47:54.891391    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4e9258_9777_4685_a622_1e7609e6c04c.slice/crio-10ba55358bf69c0c8ec992717b41ebbe03bb87355c2cdba65469f12bfb92d23a WatchSource:0}: Error finding container 10ba55358bf69c0c8ec992717b41ebbe03bb87355c2cdba65469f12bfb92d23a: Status 404 returned error can't find the container with id 10ba55358bf69c0c8ec992717b41ebbe03bb87355c2cdba65469f12bfb92d23a
Jul 21 17:47:55.053579 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:55.053490    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-140548771608592"
Jul 21 17:47:55.053733 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:55.053575    2561 scope.go:117] "RemoveContainer" containerID="f75f6052fde2cda2fee3cef7eaeb72ac60b7418588bf0a608d749c361303118b"
Jul 21 17:47:55.055430 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:55.055398    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw" event={"ID":"4a4e9258-9777-4685-a622-1e7609e6c04c","Type":"ContainerStarted","Data":"10ba55358bf69c0c8ec992717b41ebbe03bb87355c2cdba65469f12bfb92d23a"}
Jul 21 17:47:56.063046 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:56.063014    2561 generic.go:356] "Generic (PLEG): container finished" podID="4a4e9258-9777-4685-a622-1e7609e6c04c" containerID="10da18ccb0247d9b65cc1b3ee4b4990ed667afd7faafd5d4a548b6514ec06e9c" exitCode=0
Jul 21 17:47:56.063438 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:56.063072    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw" event={"ID":"4a4e9258-9777-4685-a622-1e7609e6c04c","Type":"ContainerDied","Data":"10da18ccb0247d9b65cc1b3ee4b4990ed667afd7faafd5d4a548b6514ec06e9c"}
Jul 21 17:47:57.201527 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:57.201503    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:57.282501 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:57.282453    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4e9258-9777-4685-a622-1e7609e6c04c-maas-api-service-ca\") pod \"4a4e9258-9777-4685-a622-1e7609e6c04c\" (UID: \"4a4e9258-9777-4685-a622-1e7609e6c04c\") "
Jul 21 17:47:57.282879 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:57.282857    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4e9258-9777-4685-a622-1e7609e6c04c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a4e9258-9777-4685-a622-1e7609e6c04c" (UID: "4a4e9258-9777-4685-a622-1e7609e6c04c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:47:57.383979 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:57.383891    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a4e9258-9777-4685-a622-1e7609e6c04c-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:47:58.074684 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:58.074652    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"
Jul 21 17:47:58.074854 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:58.074693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw" event={"ID":"4a4e9258-9777-4685-a622-1e7609e6c04c","Type":"ContainerDied","Data":"10ba55358bf69c0c8ec992717b41ebbe03bb87355c2cdba65469f12bfb92d23a"}
Jul 21 17:47:58.074854 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:47:58.074737    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10ba55358bf69c0c8ec992717b41ebbe03bb87355c2cdba65469f12bfb92d23a"
Jul 21 17:48:04.903018 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:04.902980    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"]
Jul 21 17:48:04.903487 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:04.903164    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" podUID="e54e8fd1-f808-4b8f-b568-c9f831d87a39" containerName="maas-api" containerID="cri-o://e23161d4e180d2b7d0284965844f04819123fd0ce9bac533b23bd625d039df12" gracePeriod=30
Jul 21 17:48:04.935026 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:04.934996    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"]
Jul 21 17:48:04.938089 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:04.938059    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6fedc9-29744265-mq4bw"]
Jul 21 17:48:05.114940 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.114910    2561 generic.go:356] "Generic (PLEG): container finished" podID="e54e8fd1-f808-4b8f-b568-c9f831d87a39" containerID="e23161d4e180d2b7d0284965844f04819123fd0ce9bac533b23bd625d039df12" exitCode=0
Jul 21 17:48:05.115091 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.114981    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" event={"ID":"e54e8fd1-f808-4b8f-b568-c9f831d87a39","Type":"ContainerDied","Data":"e23161d4e180d2b7d0284965844f04819123fd0ce9bac533b23bd625d039df12"}
Jul 21 17:48:05.156363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.156299    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:48:05.250671 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.250642    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54e8fd1-f808-4b8f-b568-c9f831d87a39-openshift-service-ca\") pod \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") "
Jul 21 17:48:05.250671 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.250677    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls\") pod \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") "
Jul 21 17:48:05.250984 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.250739    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvcpx\" (UniqueName: \"kubernetes.io/projected/e54e8fd1-f808-4b8f-b568-c9f831d87a39-kube-api-access-pvcpx\") pod \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\" (UID: \"e54e8fd1-f808-4b8f-b568-c9f831d87a39\") "
Jul 21 17:48:05.251092 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.251062    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e54e8fd1-f808-4b8f-b568-c9f831d87a39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e54e8fd1-f808-4b8f-b568-c9f831d87a39" (UID: "e54e8fd1-f808-4b8f-b568-c9f831d87a39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:48:05.252935 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.252911    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e54e8fd1-f808-4b8f-b568-c9f831d87a39" (UID: "e54e8fd1-f808-4b8f-b568-c9f831d87a39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:48:05.253047 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.253031    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54e8fd1-f808-4b8f-b568-c9f831d87a39-kube-api-access-pvcpx" (OuterVolumeSpecName: "kube-api-access-pvcpx") pod "e54e8fd1-f808-4b8f-b568-c9f831d87a39" (UID: "e54e8fd1-f808-4b8f-b568-c9f831d87a39"). InnerVolumeSpecName "kube-api-access-pvcpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:48:05.352263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.352227    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54e8fd1-f808-4b8f-b568-c9f831d87a39-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:05.352263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.352259    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54e8fd1-f808-4b8f-b568-c9f831d87a39-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:05.352263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:05.352270    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvcpx\" (UniqueName: \"kubernetes.io/projected/e54e8fd1-f808-4b8f-b568-c9f831d87a39-kube-api-access-pvcpx\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:06.127540 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.127508    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"
Jul 21 17:48:06.128033 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.127513    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf" event={"ID":"e54e8fd1-f808-4b8f-b568-c9f831d87a39","Type":"ContainerDied","Data":"44bdb76633c7844c7efb0beb09f9de0c294d12ac2fd878ac3b256ac4a44abc7e"}
Jul 21 17:48:06.128033 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.127668    2561 scope.go:117] "RemoveContainer" containerID="e23161d4e180d2b7d0284965844f04819123fd0ce9bac533b23bd625d039df12"
Jul 21 17:48:06.152247 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.152211    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"]
Jul 21 17:48:06.158915 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.158887    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6fedc9-6dd588d8d7-8wjxf"]
Jul 21 17:48:06.514358 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.514328    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf41f01-4cc8-4577-b3e3-7d9a466e582b" path="/var/lib/kubelet/pods/bbf41f01-4cc8-4577-b3e3-7d9a466e582b/volumes"
Jul 21 17:48:06.514717 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:06.514696    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e54e8fd1-f808-4b8f-b568-c9f831d87a39" path="/var/lib/kubelet/pods/e54e8fd1-f808-4b8f-b568-c9f831d87a39/volumes"
Jul 21 17:48:09.541201 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:09.541156    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"]
Jul 21 17:48:09.544858 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:09.544822    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6fedc9-tsdcw"]
Jul 21 17:48:10.515665 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:10.515631    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a4e9258-9777-4685-a622-1e7609e6c04c" path="/var/lib/kubelet/pods/4a4e9258-9777-4685-a622-1e7609e6c04c/volumes"
Jul 21 17:48:16.836020 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.835986    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"]
Jul 21 17:48:16.836425 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.836357    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e54e8fd1-f808-4b8f-b568-c9f831d87a39" containerName="maas-api"
Jul 21 17:48:16.836425 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.836368    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e54e8fd1-f808-4b8f-b568-c9f831d87a39" containerName="maas-api"
Jul 21 17:48:16.836425 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.836377    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4e9258-9777-4685-a622-1e7609e6c04c" containerName="revoke-keys"
Jul 21 17:48:16.836425 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.836383    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4e9258-9777-4685-a622-1e7609e6c04c" containerName="revoke-keys"
Jul 21 17:48:16.836551 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.836461    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4e9258-9777-4685-a622-1e7609e6c04c" containerName="revoke-keys"
Jul 21 17:48:16.836551 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.836471    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e54e8fd1-f808-4b8f-b568-c9f831d87a39" containerName="maas-api"
Jul 21 17:48:16.839732 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.839715    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:16.842264 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.842236    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dl9wf\""
Jul 21 17:48:16.848642 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.848623    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"]
Jul 21 17:48:16.958142 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:16.958100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d15027fe-103d-4086-aa32-83d5eaa60481-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh\" (UID: \"d15027fe-103d-4086-aa32-83d5eaa60481\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:17.059252 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:17.059205    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d15027fe-103d-4086-aa32-83d5eaa60481-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh\" (UID: \"d15027fe-103d-4086-aa32-83d5eaa60481\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:17.059799 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:17.059781    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d15027fe-103d-4086-aa32-83d5eaa60481-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh\" (UID: \"d15027fe-103d-4086-aa32-83d5eaa60481\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:17.150810 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:17.150780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:17.296294 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:17.296266    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"]
Jul 21 17:48:17.298370 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:48:17.298340    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15027fe_103d_4086_aa32_83d5eaa60481.slice/crio-26c95459913a7fbb5196e1c0fb4edb3cc3117bf363d9387346a0c5912373f8c0 WatchSource:0}: Error finding container 26c95459913a7fbb5196e1c0fb4edb3cc3117bf363d9387346a0c5912373f8c0: Status 404 returned error can't find the container with id 26c95459913a7fbb5196e1c0fb4edb3cc3117bf363d9387346a0c5912373f8c0
Jul 21 17:48:18.197150 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:18.197113    2561 generic.go:356] "Generic (PLEG): container finished" podID="d15027fe-103d-4086-aa32-83d5eaa60481" containerID="0f53cd9c5dce484cfd9a1bd4b41edc27c811a97e1415c2a936a7dcf4c98ff52d" exitCode=0
Jul 21 17:48:18.197547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:18.197162    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh" event={"ID":"d15027fe-103d-4086-aa32-83d5eaa60481","Type":"ContainerDied","Data":"0f53cd9c5dce484cfd9a1bd4b41edc27c811a97e1415c2a936a7dcf4c98ff52d"}
Jul 21 17:48:18.197547 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:18.197226    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh" event={"ID":"d15027fe-103d-4086-aa32-83d5eaa60481","Type":"ContainerStarted","Data":"26c95459913a7fbb5196e1c0fb4edb3cc3117bf363d9387346a0c5912373f8c0"}
Jul 21 17:48:19.344628 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:19.344602    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:19.477627 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:19.477517    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d15027fe-103d-4086-aa32-83d5eaa60481-maas-api-service-ca\") pod \"d15027fe-103d-4086-aa32-83d5eaa60481\" (UID: \"d15027fe-103d-4086-aa32-83d5eaa60481\") "
Jul 21 17:48:19.477859 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:19.477838    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15027fe-103d-4086-aa32-83d5eaa60481-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d15027fe-103d-4086-aa32-83d5eaa60481" (UID: "d15027fe-103d-4086-aa32-83d5eaa60481"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:48:19.578507 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:19.578468    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d15027fe-103d-4086-aa32-83d5eaa60481-maas-api-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:20.218113 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:20.218080    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"
Jul 21 17:48:20.218419 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:20.218078    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh" event={"ID":"d15027fe-103d-4086-aa32-83d5eaa60481","Type":"ContainerDied","Data":"26c95459913a7fbb5196e1c0fb4edb3cc3117bf363d9387346a0c5912373f8c0"}
Jul 21 17:48:20.218419 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:20.218214    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26c95459913a7fbb5196e1c0fb4edb3cc3117bf363d9387346a0c5912373f8c0"
Jul 21 17:48:27.307872 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.307794    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"]
Jul 21 17:48:27.308251 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.308018    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" podUID="e4316a9c-86e6-453b-b082-57db1e4e1bc9" containerName="maas-api" containerID="cri-o://59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24" gracePeriod=30
Jul 21 17:48:27.322156 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.322127    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"]
Jul 21 17:48:27.330137 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.330114    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5a36a6-29744265-4mspn"]
Jul 21 17:48:27.552744 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.552717    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:48:27.644114 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.644089    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8tbx\" (UniqueName: \"kubernetes.io/projected/e4316a9c-86e6-453b-b082-57db1e4e1bc9-kube-api-access-z8tbx\") pod \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") "
Jul 21 17:48:27.644247 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.644154    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls\") pod \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") "
Jul 21 17:48:27.644247 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.644204    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4316a9c-86e6-453b-b082-57db1e4e1bc9-openshift-service-ca\") pod \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\" (UID: \"e4316a9c-86e6-453b-b082-57db1e4e1bc9\") "
Jul 21 17:48:27.644673 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.644637    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4316a9c-86e6-453b-b082-57db1e4e1bc9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e4316a9c-86e6-453b-b082-57db1e4e1bc9" (UID: "e4316a9c-86e6-453b-b082-57db1e4e1bc9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 17:48:27.646324 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.646297    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4316a9c-86e6-453b-b082-57db1e4e1bc9-kube-api-access-z8tbx" (OuterVolumeSpecName: "kube-api-access-z8tbx") pod "e4316a9c-86e6-453b-b082-57db1e4e1bc9" (UID: "e4316a9c-86e6-453b-b082-57db1e4e1bc9"). InnerVolumeSpecName "kube-api-access-z8tbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 17:48:27.646438 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.646405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e4316a9c-86e6-453b-b082-57db1e4e1bc9" (UID: "e4316a9c-86e6-453b-b082-57db1e4e1bc9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 17:48:27.745592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.745541    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4316a9c-86e6-453b-b082-57db1e4e1bc9-openshift-service-ca\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:27.745592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.745588    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8tbx\" (UniqueName: \"kubernetes.io/projected/e4316a9c-86e6-453b-b082-57db1e4e1bc9-kube-api-access-z8tbx\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:27.745739 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:27.745598    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4316a9c-86e6-453b-b082-57db1e4e1bc9-maas-api-tls\") on node \"ip-10-0-140-24.ec2.internal\" DevicePath \"\""
Jul 21 17:48:28.262222 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.262185    2561 generic.go:356] "Generic (PLEG): container finished" podID="e4316a9c-86e6-453b-b082-57db1e4e1bc9" containerID="59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24" exitCode=0
Jul 21 17:48:28.262386 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.262227    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"
Jul 21 17:48:28.262386 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.262232    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" event={"ID":"e4316a9c-86e6-453b-b082-57db1e4e1bc9","Type":"ContainerDied","Data":"59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24"}
Jul 21 17:48:28.262386 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.262358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9" event={"ID":"e4316a9c-86e6-453b-b082-57db1e4e1bc9","Type":"ContainerDied","Data":"345209bc46514078cab84353ecaad952ce90cb69177ef52a0c6b23b5709a6870"}
Jul 21 17:48:28.262490 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.262387    2561 scope.go:117] "RemoveContainer" containerID="59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24"
Jul 21 17:48:28.272240 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.272224    2561 scope.go:117] "RemoveContainer" containerID="59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24"
Jul 21 17:48:28.272476 ip-10-0-140-24 kubenswrapper[2561]: E0721 17:48:28.272458    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24\": container with ID starting with 59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24 not found: ID does not exist" containerID="59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24"
Jul 21 17:48:28.272531 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.272486    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24"} err="failed to get container status \"59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24\": rpc error: code = NotFound desc = could not find container \"59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24\": container with ID starting with 59689b96b82e4bb40f50d1ef19719620d4d2adcf5027758c8ec5fb19091ecb24 not found: ID does not exist"
Jul 21 17:48:28.284430 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.284409    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"]
Jul 21 17:48:28.287207 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.287189    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5a36a6-c48bfbdcc-n2hp9"]
Jul 21 17:48:28.512883 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.512821    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca790d98-774c-4ccd-b78b-8f2dc28b32c4" path="/var/lib/kubelet/pods/ca790d98-774c-4ccd-b78b-8f2dc28b32c4/volumes"
Jul 21 17:48:28.513183 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:28.513145    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4316a9c-86e6-453b-b082-57db1e4e1bc9" path="/var/lib/kubelet/pods/e4316a9c-86e6-453b-b082-57db1e4e1bc9/volumes"
Jul 21 17:48:31.940056 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:31.940019    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"]
Jul 21 17:48:31.942941 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:31.942917    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5a36a6-q8dsh"]
Jul 21 17:48:32.513711 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:32.513678    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d15027fe-103d-4086-aa32-83d5eaa60481" path="/var/lib/kubelet/pods/d15027fe-103d-4086-aa32-83d5eaa60481/volumes"
Jul 21 17:48:45.766300 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:45.766266    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-csxsd_648562a2-3465-4e0a-88f9-0968d6bf7ec2/manager/0.log"
Jul 21 17:48:46.140871 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:46.140789    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7_e9cc9529-59b5-49a2-a7b5-a640d9bfbc11/copy-manifests/0.log"
Jul 21 17:48:46.147989 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:46.147960    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7_e9cc9529-59b5-49a2-a7b5-a640d9bfbc11/manager/0.log"
Jul 21 17:48:46.713836 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:46.713806    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744250-fvxdw_0621c7ce-5544-4b6f-9460-a9ea5a9a85f0/cleanup/0.log"
Jul 21 17:48:46.825878 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:46.825851    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744265-hx6lm_a8c39a9d-d516-40a3-8d55-2f3f2ce3576c/cleanup/0.log"
Jul 21 17:48:47.693239 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.693181    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb_8e055721-2549-419e-974e-bc59e120c7db/pull/0.log"
Jul 21 17:48:47.700420 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.700370    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb_8e055721-2549-419e-974e-bc59e120c7db/extract/0.log"
Jul 21 17:48:47.707386 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.707367    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb_8e055721-2549-419e-974e-bc59e120c7db/util/0.log"
Jul 21 17:48:47.820643 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.820613    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b_f59109e5-f725-4588-b6eb-fb632500afb5/util/0.log"
Jul 21 17:48:47.827401 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.827369    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b_f59109e5-f725-4588-b6eb-fb632500afb5/pull/0.log"
Jul 21 17:48:47.834177 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.834153    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b_f59109e5-f725-4588-b6eb-fb632500afb5/extract/0.log"
Jul 21 17:48:47.938893 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.938874    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl_6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8/util/0.log"
Jul 21 17:48:47.945662 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.945578    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl_6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8/pull/0.log"
Jul 21 17:48:47.954593 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:47.954548    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl_6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8/extract/0.log"
Jul 21 17:48:48.062517 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:48.062492    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l_f7e56816-ddf7-4baf-8dae-961e2b0ec61f/util/0.log"
Jul 21 17:48:48.069604 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:48.069583    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l_f7e56816-ddf7-4baf-8dae-961e2b0ec61f/pull/0.log"
Jul 21 17:48:48.076244 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:48.076194    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l_f7e56816-ddf7-4baf-8dae-961e2b0ec61f/extract/0.log"
Jul 21 17:48:48.316946 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:48.316860    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jmxcc_1a3265e6-7cf3-42e4-b485-53979720a590/manager/0.log"
Jul 21 17:48:48.661629 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:48.661603    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vzkmm_1a2a4b3d-6789-4781-b127-d8aaebfac6a1/registry-server/0.log"
Jul 21 17:48:48.780146 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:48.780118    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6zptj_07e45df4-78c2-4c31-9da0-e9abece95f6b/manager/0.log"
Jul 21 17:48:49.003191 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:49.003128    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fktzx_b74262ec-7b72-4a58-a673-0316ebfadf0f/manager/0.log"
Jul 21 17:48:49.468807 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:49.468776    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdp2p_1c3fec08-4d01-4c93-be47-906d0fe10b1b/discovery/0.log"
Jul 21 17:48:49.581745 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:49.581715    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66868cbd5-dl2ln_59a3d206-484b-453e-aafd-d0d2936c2989/kube-auth-proxy/0.log"
Jul 21 17:48:50.051531 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:50.051497    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-m2q87_15fe5898-8eb0-49fe-9b23-1e14a01e253c/payload-processing/0.log"
Jul 21 17:48:50.615773 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:50.615745    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-n4dqc_f520906f-c8e5-4cbb-adb6-a80ce36f1431/main/0.log"
Jul 21 17:48:50.944830 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:50.944784    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-zqbcg_2b4321cc-29f2-4bce-99be-1cd709c78cb9/main/0.log"
Jul 21 17:48:56.951576 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.951524    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b76jr/must-gather-ndcvl"]
Jul 21 17:48:56.951982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.951941    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4316a9c-86e6-453b-b082-57db1e4e1bc9" containerName="maas-api"
Jul 21 17:48:56.951982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.951954    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4316a9c-86e6-453b-b082-57db1e4e1bc9" containerName="maas-api"
Jul 21 17:48:56.951982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.951961    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d15027fe-103d-4086-aa32-83d5eaa60481" containerName="revoke-keys"
Jul 21 17:48:56.951982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.951967    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d15027fe-103d-4086-aa32-83d5eaa60481" containerName="revoke-keys"
Jul 21 17:48:56.952124 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.952029    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4316a9c-86e6-453b-b082-57db1e4e1bc9" containerName="maas-api"
Jul 21 17:48:56.952124 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.952038    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d15027fe-103d-4086-aa32-83d5eaa60481" containerName="revoke-keys"
Jul 21 17:48:56.955241 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.955225    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:56.960968 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.960938    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b76jr\"/\"default-dockercfg-4vv7m\""
Jul 21 17:48:56.961888 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.961865    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b76jr\"/\"openshift-service-ca.crt\""
Jul 21 17:48:56.962855 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.961880    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b76jr\"/\"kube-root-ca.crt\""
Jul 21 17:48:56.966624 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.966577    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b76jr/must-gather-ndcvl"]
Jul 21 17:48:56.999741 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.999711    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx4x4\" (UniqueName: \"kubernetes.io/projected/c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87-kube-api-access-zx4x4\") pod \"must-gather-ndcvl\" (UID: \"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87\") " pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:56.999886 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:56.999775    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87-must-gather-output\") pod \"must-gather-ndcvl\" (UID: \"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87\") " pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:57.101260 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:57.101223    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx4x4\" (UniqueName: \"kubernetes.io/projected/c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87-kube-api-access-zx4x4\") pod \"must-gather-ndcvl\" (UID: \"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87\") " pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:57.101430 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:57.101304    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87-must-gather-output\") pod \"must-gather-ndcvl\" (UID: \"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87\") " pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:57.101676 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:57.101657    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87-must-gather-output\") pod \"must-gather-ndcvl\" (UID: \"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87\") " pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:57.110219 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:57.110193    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx4x4\" (UniqueName: \"kubernetes.io/projected/c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87-kube-api-access-zx4x4\") pod \"must-gather-ndcvl\" (UID: \"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87\") " pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:57.271705 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:57.271610    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b76jr/must-gather-ndcvl"
Jul 21 17:48:57.424639 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:57.424601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b76jr/must-gather-ndcvl"]
Jul 21 17:48:57.427395 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:48:57.427364    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5f7d5fb_750e_4fb3_8ee5_b7b28e5f1c87.slice/crio-f65ef5b56c8c4bdd9cbb48a826a4a9b771ea0b04f05c64154916bad7cb297cc6 WatchSource:0}: Error finding container f65ef5b56c8c4bdd9cbb48a826a4a9b771ea0b04f05c64154916bad7cb297cc6: Status 404 returned error can't find the container with id f65ef5b56c8c4bdd9cbb48a826a4a9b771ea0b04f05c64154916bad7cb297cc6
Jul 21 17:48:58.439647 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:58.439619    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b76jr/must-gather-ndcvl" event={"ID":"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87","Type":"ContainerStarted","Data":"d70dcb79c0155390088305bca354d3ec7a0e4396cef45b0443c754349aeb344a"}
Jul 21 17:48:58.439927 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:58.439664    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b76jr/must-gather-ndcvl" event={"ID":"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87","Type":"ContainerStarted","Data":"f65ef5b56c8c4bdd9cbb48a826a4a9b771ea0b04f05c64154916bad7cb297cc6"}
Jul 21 17:48:59.453659 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:59.453608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b76jr/must-gather-ndcvl" event={"ID":"c5f7d5fb-750e-4fb3-8ee5-b7b28e5f1c87","Type":"ContainerStarted","Data":"39e16fa6eb5662fd5e409bf540bd9065f1de871081d0f1ee566f73e40019edd3"}
Jul 21 17:48:59.473832 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:59.473784    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b76jr/must-gather-ndcvl" podStartSLOduration=2.611701226 podStartE2EDuration="3.473766156s" totalImagesPullingTime="862.06493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:48:59.471958005 +0000 UTC m=+2819.436857610" watchObservedRunningTime="2026-07-21 17:48:59.473766156 +0000 UTC m=+2819.438665763"
Jul 21 17:48:59.987113 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:48:59.987084    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jjmrd_0c24b7c8-f986-4410-bf06-5d31d80f43b6/global-pull-secret-syncer/0.log"
Jul 21 17:49:00.090918 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:00.090884    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r9ckt_dd8e5c13-07a9-49bd-bf9f-08de8be07870/konnectivity-agent/0.log"
Jul 21 17:49:00.203434 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:00.203405    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-24.ec2.internal_de9305d86924dd928dcf28542838bdc5/haproxy/0.log"
Jul 21 17:49:04.397040 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.396993    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb_8e055721-2549-419e-974e-bc59e120c7db/extract/0.log"
Jul 21 17:49:04.427962 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.427933    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb_8e055721-2549-419e-974e-bc59e120c7db/util/0.log"
Jul 21 17:49:04.462539 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.462506    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pq6nb_8e055721-2549-419e-974e-bc59e120c7db/pull/0.log"
Jul 21 17:49:04.503206 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.503181    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b_f59109e5-f725-4588-b6eb-fb632500afb5/extract/0.log"
Jul 21 17:49:04.531992 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.531961    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b_f59109e5-f725-4588-b6eb-fb632500afb5/util/0.log"
Jul 21 17:49:04.564743 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.564710    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lxj4b_f59109e5-f725-4588-b6eb-fb632500afb5/pull/0.log"
Jul 21 17:49:04.598664 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.598567    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl_6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8/extract/0.log"
Jul 21 17:49:04.625543 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.625518    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl_6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8/util/0.log"
Jul 21 17:49:04.652151 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.651898    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cx4zl_6c7a5b31-bd95-4d61-b9ee-c41973b4bfc8/pull/0.log"
Jul 21 17:49:04.696363 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.696335    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l_f7e56816-ddf7-4baf-8dae-961e2b0ec61f/extract/0.log"
Jul 21 17:49:04.723362 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.723337    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l_f7e56816-ddf7-4baf-8dae-961e2b0ec61f/util/0.log"
Jul 21 17:49:04.749139 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:04.749090    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rk26l_f7e56816-ddf7-4baf-8dae-961e2b0ec61f/pull/0.log"
Jul 21 17:49:05.403594 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:05.403492    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jmxcc_1a3265e6-7cf3-42e4-b485-53979720a590/manager/0.log"
Jul 21 17:49:05.532528 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:05.532477    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vzkmm_1a2a4b3d-6789-4781-b127-d8aaebfac6a1/registry-server/0.log"
Jul 21 17:49:05.738496 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:05.738420    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6zptj_07e45df4-78c2-4c31-9da0-e9abece95f6b/manager/0.log"
Jul 21 17:49:05.856618 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:05.856577    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fktzx_b74262ec-7b72-4a58-a673-0316ebfadf0f/manager/0.log"
Jul 21 17:49:07.794689 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:07.794643    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-t9rs5_90b3084e-d2d0-47f2-858e-64683244d001/kube-state-metrics/0.log"
Jul 21 17:49:07.820062 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:07.820024    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-t9rs5_90b3084e-d2d0-47f2-858e-64683244d001/kube-rbac-proxy-main/0.log"
Jul 21 17:49:07.845982 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:07.845944    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-t9rs5_90b3084e-d2d0-47f2-858e-64683244d001/kube-rbac-proxy-self/0.log"
Jul 21 17:49:07.877432 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:07.877396    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-9f77db875-9mvj4_0b33d9fe-25d2-4825-af17-9e51cf0fe35a/metrics-server/0.log"
Jul 21 17:49:07.950292 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:07.950262    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44b96_85e079b2-abd1-4fdf-b22a-28f4e78c2a26/node-exporter/0.log"
Jul 21 17:49:07.978111 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:07.978079    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44b96_85e079b2-abd1-4fdf-b22a-28f4e78c2a26/kube-rbac-proxy/0.log"
Jul 21 17:49:08.007132 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:08.007094    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44b96_85e079b2-abd1-4fdf-b22a-28f4e78c2a26/init-textfile/0.log"
Jul 21 17:49:08.573797 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:08.573758    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-68zmz_086997e7-6677-46f5-be2b-b7852e9bb70a/prometheus-operator-admission-webhook/0.log"
Jul 21 17:49:08.612026 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:08.611996    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6df97fbddb-wts2s_a7100a61-1e0a-44b9-bc6c-46827d4e6ae8/telemeter-client/0.log"
Jul 21 17:49:08.637084 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:08.637054    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6df97fbddb-wts2s_a7100a61-1e0a-44b9-bc6c-46827d4e6ae8/reload/0.log"
Jul 21 17:49:08.662053 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:08.662025    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6df97fbddb-wts2s_a7100a61-1e0a-44b9-bc6c-46827d4e6ae8/kube-rbac-proxy/0.log"
Jul 21 17:49:10.008518 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.008488    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"]
Jul 21 17:49:10.016424 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.016391    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.021362 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.021325    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"]
Jul 21 17:49:10.038592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.038421    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-proc\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.038592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.038515    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-podres\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.038592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.038589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-sys\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.038881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.038616    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-lib-modules\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.038881 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.038684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjr2n\" (UniqueName: \"kubernetes.io/projected/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-kube-api-access-gjr2n\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.140268 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140226    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjr2n\" (UniqueName: \"kubernetes.io/projected/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-kube-api-access-gjr2n\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.140622 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140597    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-proc\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.140772 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140755    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-podres\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.140928 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-sys\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.141055 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.141040    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-lib-modules\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.141349 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140711    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-proc\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.141475 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140962    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-sys\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.141590 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.140908    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-podres\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.141691 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.141248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-lib-modules\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.156009 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.155954    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjr2n\" (UniqueName: \"kubernetes.io/projected/ff338d5b-d688-4ecc-8ee2-b1ea493cc632-kube-api-access-gjr2n\") pod \"perf-node-gather-daemonset-4fg6p\" (UID: \"ff338d5b-d688-4ecc-8ee2-b1ea493cc632\") " pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.330592 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.330269    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:10.524694 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.524668    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"]
Jul 21 17:49:10.527054 ip-10-0-140-24 kubenswrapper[2561]: W0721 17:49:10.527022    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff338d5b_d688_4ecc_8ee2_b1ea493cc632.slice/crio-47def57c154184ad10c3557b2ac4ff5509f0785e573f54c2735293164a0b6236 WatchSource:0}: Error finding container 47def57c154184ad10c3557b2ac4ff5509f0785e573f54c2735293164a0b6236: Status 404 returned error can't find the container with id 47def57c154184ad10c3557b2ac4ff5509f0785e573f54c2735293164a0b6236
Jul 21 17:49:10.560309 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:10.559949    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p" event={"ID":"ff338d5b-d688-4ecc-8ee2-b1ea493cc632","Type":"ContainerStarted","Data":"47def57c154184ad10c3557b2ac4ff5509f0785e573f54c2735293164a0b6236"}
Jul 21 17:49:11.191454 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:11.191423    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-567cb48fcd-2xknw_cd1b239a-9fd3-481a-a26f-2429019815f7/console/0.log"
Jul 21 17:49:11.569772 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:11.569679    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p" event={"ID":"ff338d5b-d688-4ecc-8ee2-b1ea493cc632","Type":"ContainerStarted","Data":"4f0da1c3e3775165715ee9cf47c8a4b760f5936259372b09d1dc075d60ceff74"}
Jul 21 17:49:11.569953 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:11.569815    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:11.589112 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:11.589057    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p" podStartSLOduration=2.589040517 podStartE2EDuration="2.589040517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:49:11.58541389 +0000 UTC m=+2831.550313494" watchObservedRunningTime="2026-07-21 17:49:11.589040517 +0000 UTC m=+2831.553940118"
Jul 21 17:49:12.916102 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:12.916065    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2pcgv_9b75e216-b714-4036-8214-ca7adddb4839/dns/0.log"
Jul 21 17:49:12.942417 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:12.942388    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2pcgv_9b75e216-b714-4036-8214-ca7adddb4839/kube-rbac-proxy/0.log"
Jul 21 17:49:13.131769 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:13.131740    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kns2f_e4892ce1-b232-4ced-b142-289d7804ecff/dns-node-resolver/0.log"
Jul 21 17:49:13.707329 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:13.707302    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b8bb8d44-wvlbf_100e03cb-0ee7-4983-810d-0906466049c2/registry/0.log"
Jul 21 17:49:13.760106 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:13.760072    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7k5q8_c65d0345-d0cf-40c2-9bd7-0086726a8f31/node-ca/0.log"
Jul 21 17:49:14.946999 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:14.946968    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdp2p_1c3fec08-4d01-4c93-be47-906d0fe10b1b/discovery/0.log"
Jul 21 17:49:14.970268 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:14.970237    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66868cbd5-dl2ln_59a3d206-484b-453e-aafd-d0d2936c2989/kube-auth-proxy/0.log"
Jul 21 17:49:15.144086 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:15.144058    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-m2q87_15fe5898-8eb0-49fe-9b23-1e14a01e253c/payload-processing/0.log"
Jul 21 17:49:15.935282 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:15.935236    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-grf9q_1868e074-fca4-47b5-a4e2-87e2eb73923d/serve-healthcheck-canary/0.log"
Jul 21 17:49:16.477025 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:16.476999    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gmllb_24734282-d9c9-411e-b08b-748d83bc3fe4/kube-rbac-proxy/0.log"
Jul 21 17:49:16.501478 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:16.501453    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gmllb_24734282-d9c9-411e-b08b-748d83bc3fe4/exporter/0.log"
Jul 21 17:49:16.527437 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:16.527413    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gmllb_24734282-d9c9-411e-b08b-748d83bc3fe4/extractor/0.log"
Jul 21 17:49:17.590728 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:17.590696    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b76jr/perf-node-gather-daemonset-4fg6p"
Jul 21 17:49:19.523048 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:19.523018    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-csxsd_648562a2-3465-4e0a-88f9-0968d6bf7ec2/manager/0.log"
Jul 21 17:49:19.774754 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:19.774714    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7_e9cc9529-59b5-49a2-a7b5-a640d9bfbc11/manager/0.log"
Jul 21 17:49:19.801743 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:19.801707    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cc55bc8b7-mtpn7_e9cc9529-59b5-49a2-a7b5-a640d9bfbc11/copy-manifests/0.log"
Jul 21 17:49:20.489121 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:20.489087    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdp2p_1c3fec08-4d01-4c93-be47-906d0fe10b1b/discovery/0.log"
Jul 21 17:49:20.500699 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:20.500672    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66868cbd5-dl2ln_59a3d206-484b-453e-aafd-d0d2936c2989/kube-auth-proxy/0.log"
Jul 21 17:49:20.617930 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:20.617840    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-m2q87_15fe5898-8eb0-49fe-9b23-1e14a01e253c/payload-processing/0.log"
Jul 21 17:49:21.200443 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:21.200403    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2qng9_8d548995-f3ec-4fda-bcd4-b9418d8a37c9/openshift-lws-operator/0.log"
Jul 21 17:49:22.049263 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:22.049226    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gdp2p_1c3fec08-4d01-4c93-be47-906d0fe10b1b/discovery/0.log"
Jul 21 17:49:22.070067 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:22.070027    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66868cbd5-dl2ln_59a3d206-484b-453e-aafd-d0d2936c2989/kube-auth-proxy/0.log"
Jul 21 17:49:22.192041 ip-10-0-140-24 kubenswrapper[2561]: I0721 17:49:22.192009    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-m2q87_15fe5898-8eb0-49fe-9b23-1e14a01e253c/payload-processing/0.log"