Jul 23 19:16:24.602956 ip-10-0-140-14 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 19:16:24.958929 ip-10-0-140-14 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 19:16:24.958929 ip-10-0-140-14 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 19:16:24.958929 ip-10-0-140-14 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 19:16:24.958929 ip-10-0-140-14 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 19:16:24.958929 ip-10-0-140-14 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 19:16:24.961202 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.961057    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 19:16:24.965651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965634    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:16:24.965651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965652    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965657    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965662    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965665    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965681    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965684    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965687    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965690    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965693    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965696    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965700    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965703    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965712    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965715    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965718    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965721    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965724    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965727    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965729    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:16:24.965728 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965732    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965735    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965738    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965741    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965744    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965746    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965749    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965751    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965753    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965756    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965758    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965761    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965765    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965768    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965770    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965773    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965775    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965778    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965781    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965783    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:16:24.966184 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965786    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965788    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965790    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965793    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965795    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965798    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965800    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965803    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965805    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965808    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965810    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965812    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965815    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965817    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965821    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965824    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965828    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965831    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965834    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:16:24.966651 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965836    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965839    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965842    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965844    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965846    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965849    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965852    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965854    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965856    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965859    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965861    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965864    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965867    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965869    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965871    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965874    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965876    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965878    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965881    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965883    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:16:24.967140 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965885    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965888    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965890    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965892    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965895    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965897    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965900    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965903    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965907    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965910    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965912    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965915    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965917    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965920    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965922    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965925    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965927    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965930    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965933    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.965935    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:16:24.967610 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966700    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966707    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966711    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966714    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966717    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966721    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966724    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966727    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966730    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966734    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966737    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966740    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966743    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966746    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966748    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966751    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966754    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966757    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966759    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:16:24.968146 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966763    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966766    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966769    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966773    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966775    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966778    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966781    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966783    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966785    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966789    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966792    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966795    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966797    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966800    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966802    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966805    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966807    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966811    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966814    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:16:24.968605 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966819    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966822    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966825    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966827    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966830    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966833    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966835    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966838    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966840    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966842    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966845    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966847    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966850    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966856    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966859    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966861    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966864    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966866    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966869    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:16:24.969131 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966871    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966874    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966876    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966879    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966881    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966884    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966886    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966889    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966891    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966894    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966896    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966899    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966901    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966904    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966908    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966910    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966913    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966915    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966917    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966920    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:16:24.969595 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966922    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966926    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966929    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966931    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966934    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966936    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966941    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966943    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966946    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966949    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966951    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966954    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966956    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966959    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966961    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966963    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966966    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966968    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966971    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966973    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966976    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:16:24.970080 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.966978    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967335    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967340    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967343    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967345    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967349    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967351    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967354    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967356    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967359    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967361    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967364    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967367    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967370    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967372    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967375    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967377    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967382    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967385    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967388    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:16:24.970589 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967390    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967393    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967395    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967398    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967401    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967404    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967406    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967409    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967411    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967414    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967416    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967419    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967421    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967424    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967426    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967429    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967431    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967435    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967439    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967442    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:16:24.971077 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967445    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967447    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967450    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967452    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967455    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967457    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967460    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967462    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967464    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967469    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967472    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967474    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967477    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967479    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967481    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967484    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967486    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967491    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967494    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:16:24.971550 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967497    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967500    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967502    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967505    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967508    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967510    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967513    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967515    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967518    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967521    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967523    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967526    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967529    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967531    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967534    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967536    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967539    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967542    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967544    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967547    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:16:24.972019 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967549    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967552    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967556    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967559    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967561    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967564    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967566    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967569    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967571    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967573    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967576    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967578    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967581    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967583    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967586    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967589    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967593    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967595    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967598    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:16:24.972604 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967601    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.967603    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967689    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967696    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967703    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967707    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967711    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967714    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967719    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967723    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967727    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967730    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967733    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967736    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967739    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967743    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967746    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967749    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967751    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967754    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967759    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967762    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967765    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967767    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 19:16:24.973210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967771    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967774    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967777    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967780    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967783    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967786    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967789    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967792    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967795    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967798    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967802    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967804    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967807    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967810    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967812    2566 flags.go:64] FLAG: --enable-server="true"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967816    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967820    2566 flags.go:64] FLAG: --event-burst="100"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967823    2566 flags.go:64] FLAG: --event-qps="50"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967825    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967828    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967831    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967834    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967837    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967841    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967844    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 23 19:16:24.973798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967847    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967850    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967853    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967856    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967859    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967861    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967864    2566 flags.go:64] FLAG: --feature-gates=""
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967868    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967871    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967873    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967876    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967880    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967882    2566 flags.go:64] FLAG: --help="false"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967885    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-140-14.ec2.internal"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967889    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967892    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967894    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967898    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967901    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967904    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967906    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967909    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967912    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 19:16:24.974412 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967915    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967918    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967921    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967924    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967926    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967929    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967933    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967936    2566 flags.go:64] FLAG: --lock-file=""
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967939    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967942    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967945    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967950    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967953    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967956    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967958    2566 flags.go:64] FLAG: --logging-format="text"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967961    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967964    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967967    2566 flags.go:64] FLAG: --manifest-url=""
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967970    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967975    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967978    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967982    2566 flags.go:64] FLAG: --max-pods="110"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967985    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967987    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967990    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 19:16:24.975029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967993    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.967996    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968000    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968002    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968009    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968012    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968015    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968018    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968022    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968027    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968030    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968033    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968036    2566 flags.go:64] FLAG: --port="10250"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968039    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968041    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f8228aa373dd5c11"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968045    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968047    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968051    2566 flags.go:64] FLAG: --register-node="true"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968053    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968057    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968059    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968062    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968065    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968068    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968071    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 19:16:24.975636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968074    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968076    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968079    2566 flags.go:64] FLAG: --runonce="false"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968082    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968084    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968087    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968090    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968093    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968095    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968098    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968101    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968104    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968107    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968109    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968112    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968115    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968118    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968121    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968125    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968128    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968131    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968138    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968141    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968144    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968147    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 19:16:24.976251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968150    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968152    2566 flags.go:64] FLAG: --v="2"
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968156    2566 flags.go:64] FLAG: --version="false"
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968160    2566 flags.go:64] FLAG: --vmodule=""
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968164    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.968167    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968283    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968288    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968291    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968294    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968297    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968300    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968302    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968305    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968308    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968310    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968313    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968315    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968318    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968320    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:16:24.976851 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968323    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968326    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968328    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968330    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968333    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968336    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968338    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968340    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968345    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968348    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968351    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968353    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968355    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968358    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968360    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968363    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968365    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968367    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968370    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968373    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:16:24.977334 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968375    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968377    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968380    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968382    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968386    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968389    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968391    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968394    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968396    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968400    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968403    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968406    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968409    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968412    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968415    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968417    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968420    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968423    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968425    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968427    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:16:24.978132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968433    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968435    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968438    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968440    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968442    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968445    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968448    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968451    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968453    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968456    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968458    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968461    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968463    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968465    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968468    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968470    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968473    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968475    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968478    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968480    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:16:24.978824 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968482    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968485    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968487    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968490    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968492    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968494    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968497    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968499    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968502    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968504    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968507    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968509    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968514    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968516    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968519    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968521    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968524    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968526    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968529    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968531    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:16:24.979303 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968534    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968536    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968539    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968541    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.968543    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.969289    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.976365    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.976381    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.976403    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.976409    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976463    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976467    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976471    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976474    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976477    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:16:24.980009 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976480    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976483    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976486    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976489    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976493    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976496    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976499    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976502    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976504    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976507    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976510    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976512    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976515    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976518    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976520    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976523    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976525    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976528    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976530    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:16:24.980398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976533    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976535    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976537    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976541    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976557    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976562    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976565    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976568    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976571    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976575    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976579    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976582    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976584    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976587    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976590    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976593    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976596    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976598    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976601    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:16:24.980881 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976604    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976606    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976609    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976611    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976614    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976616    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976619    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976621    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976623    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976626    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976629    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976631    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976634    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976636    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976639    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976641    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976644    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976647    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976649    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976652    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:16:24.981350 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976655    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976657    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976659    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976662    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976664    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976667    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976685    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976687    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976690    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976692    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976695    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976698    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976700    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976703    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976705    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976708    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976711    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976713    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976716    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976718    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:16:24.981870 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976720    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976723    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976725    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976728    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976730    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976733    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976735    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976738    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976741    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976743    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976745    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976749    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976753    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976755    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976758    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:16:24.982475 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976760    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.976766    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976875    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976880    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976883    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976887    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976889    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976892    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976895    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976897    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976900    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976902    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976905    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976907    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976910    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 19:16:24.982860 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976912    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976915    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976917    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976919    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976922    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976925    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976927    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976929    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976932    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976934    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976936    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976940    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976944    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976946    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976949    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976952    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976954    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976957    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976960    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 19:16:24.983219 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976962    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976964    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976967    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976970    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976973    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976977    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976980    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976983    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976985    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976988    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976991    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976994    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976996    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.976999    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977002    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977004    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977006    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977009    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977012    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 19:16:24.983706 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977014    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977016    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977019    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977021    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977024    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977026    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977029    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977031    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977033    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977036    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977039    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977041    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977043    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977046    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977048    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977051    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977053    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977056    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977058    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977060    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 19:16:24.984155 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977063    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977065    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977068    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977070    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977072    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977075    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977077    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977080    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977082    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977084    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977087    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977090    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977092    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977094    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977097    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977099    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977101    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977104    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977107    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977109    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 19:16:24.984629 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977112    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977115    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977117    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977120    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977122    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977125    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977127    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:24.977130    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.977135    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.979870    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.981994    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.983314    2566 server.go:1019] "Starting client certificate rotation"
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.983410    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 19:16:24.985206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.983442    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 19:16:24.985877 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.985863    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 19:16:24.987080 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.987060    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 19:16:24.997897 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:24.997877    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 23 19:16:25.001381 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.001364    2566 log.go:25] "Validated CRI v1 image API"
Jul 23 19:16:25.002263 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.002227    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 19:16:25.006010 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.005992    2566 fs.go:135] Filesystem UUIDs: map[0eda0aea-c4ab-422c-8d18-7b0740926ab8:/dev/nvme0n1p3 74eb1f74-918b-4f9f-be9a-a0bdf81d21fc:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 23 19:16:25.006077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.006010    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 19:16:25.011536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.011427    2566 manager.go:216] Machine: {Timestamp:2026-07-23 19:16:25.010251049 +0000 UTC m=+0.298850632 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093776 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23ad8d647457aa22d7d40bde3841d6 SystemUUID:ec23ad8d-6474-57aa-22d7-d40bde3841d6 BootID:6a054d71-d9d2-452b-b562-bb720a3ee218 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:de:e7:87:e8:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:de:e7:87:e8:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:6a:40:b7:a3:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 19:16:25.011536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.011534    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 19:16:25.011641 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.011614    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 19:16:25.012630 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.012607    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 19:16:25.012791 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.012633    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 19:16:25.013432 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.013422    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 19:16:25.013463 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.013434    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 19:16:25.013463 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.013442    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 19:16:25.014131 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.014120    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 19:16:25.015856 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.015846    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 19:16:25.019246 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019233    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 19:16:25.019379 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019369    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 19:16:25.019409 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019401    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 19:16:25.019435 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019422    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 19:16:25.019435 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019433    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 19:16:25.019488 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019442    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 23 19:16:25.019488 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019453    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 19:16:25.019857 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019846    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 19:16:25.019917 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.019864    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 19:16:25.021309 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.021266    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 19:16:25.023028 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.023008    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 19:16:25.025228 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.025215    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 19:16:25.025270 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.025245    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 19:16:25.026365 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026352    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 19:16:25.026398 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026373    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 19:16:25.026398 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026382    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 19:16:25.026398 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026391    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026403    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026412    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026424    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026432    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026439    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026444    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026453    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026458    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 19:16:25.026479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026482    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 19:16:25.026716 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.026487    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 19:16:25.029762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.029749    2566 server.go:1268] "Started kubelet"
Jul 23 19:16:25.029867 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.029839    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 19:16:25.029922 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.029856    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 19:16:25.029967 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.029935    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 19:16:25.030532 ip-10-0-140-14 systemd[1]: Started Kubernetes Kubelet.
Jul 23 19:16:25.031905 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.031880    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 19:16:25.032011 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.031903    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 19:16:25.032011 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.031956    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 19:16:25.032287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.032269    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 19:16:25.032347 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.032298    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 19:16:25.035266 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.035238    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 19:16:25.037436 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.036455    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-14.ec2.internal.18c5013c1daba52e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-14.ec2.internal,UID:ip-10-0-140-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-14.ec2.internal,},FirstTimestamp:2026-07-23 19:16:25.029723438 +0000 UTC m=+0.318323019,LastTimestamp:2026-07-23 19:16:25.029723438 +0000 UTC m=+0.318323019,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-14.ec2.internal,}"
Jul 23 19:16:25.039191 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039173    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 19:16:25.039191 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039189    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 19:16:25.039513 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.039351    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039905    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039915    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039950    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039969    2566 factory.go:55] Registering systemd factory
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.039984    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.040060    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 19:16:25.041012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.040070    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 19:16:25.041646 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.041630    2566 factory.go:153] Registering CRI-O factory
Jul 23 19:16:25.041762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.041752    2566 factory.go:223] Registration of the crio container factory successfully
Jul 23 19:16:25.041875 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.041864    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 19:16:25.041973 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.041964    2566 factory.go:103] Registering Raw factory
Jul 23 19:16:25.042056 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.042048    2566 manager.go:1196] Started watching for new ooms in manager
Jul 23 19:16:25.042792 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.042778    2566 manager.go:349] Starting recovery of all containers
Jul 23 19:16:25.044957 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.044929    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 19:16:25.045054 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.044980    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 19:16:25.053497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.053380    2566 manager.go:354] Recovery completed
Jul 23 19:16:25.057843 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.057831    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.058863 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.058843    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tdcdd"
Jul 23 19:16:25.059864 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.059845    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.059940 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.059882    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.059940 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.059896    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.060401 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.060384    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 19:16:25.060401 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.060398    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 19:16:25.060514 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.060414    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 19:16:25.060514 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.060431    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.061404 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.061387    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.061493 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.061420    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.061493 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.061434    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.061612 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.061599    2566 policy_none.go:49] "None policy: Start"
Jul 23 19:16:25.061654 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.061619    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 19:16:25.061654 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.061630    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 19:16:25.062701 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.062689    2566 policy_none.go:47] "Start"
Jul 23 19:16:25.062802 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.062733    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-14.ec2.internal.18c5013c1f7789bc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-14.ec2.internal,UID:ip-10-0-140-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-14.ec2.internal,},FirstTimestamp:2026-07-23 19:16:25.059862972 +0000 UTC m=+0.348462574,LastTimestamp:2026-07-23 19:16:25.059862972 +0000 UTC m=+0.348462574,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-14.ec2.internal,}"
Jul 23 19:16:25.070771 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.070756    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tdcdd"
Jul 23 19:16:25.072820 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.072756    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-14.ec2.internal.18c5013c1f77ed80  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-14.ec2.internal,UID:ip-10-0-140-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-14.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-14.ec2.internal,},FirstTimestamp:2026-07-23 19:16:25.059888512 +0000 UTC m=+0.348488098,LastTimestamp:2026-07-23 19:16:25.059888512 +0000 UTC m=+0.348488098,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-14.ec2.internal,}"
Jul 23 19:16:25.098270 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098252    2566 manager.go:337] "Starting Device Plugin manager"
Jul 23 19:16:25.098372 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.098286    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 19:16:25.098372 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098296    2566 server.go:87] "Starting device plugin registration server"
Jul 23 19:16:25.098536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098525    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 19:16:25.098598 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098537    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 19:16:25.098778 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098662    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 19:16:25.098858 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098786    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 19:16:25.098858 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.098794    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 19:16:25.099284 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.099265    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 19:16:25.099364 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.099318    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.198998 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.198971    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.201764 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.201744    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.201880 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.201776    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.201880 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.201789    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.201880 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.201822    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.214712 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.214639    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.214712 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.214695    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-14.ec2.internal\": node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.230830 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.230797    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 19:16:25.232114 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.232090    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 19:16:25.232211 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.232135    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 19:16:25.232211 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.232156    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 19:16:25.232211 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.232196    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 19:16:25.237483 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.237458    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:16:25.244894 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.244874    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.332662 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.332613    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"]
Jul 23 19:16:25.332825 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.332750    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.334497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.334481    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.334560 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.334510    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.334560 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.334520    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.335805 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.335794    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.335910 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.335894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.335954 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.335926    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.336550 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.336529    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.336550 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.336541    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.336730 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.336557    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.336730 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.336564    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.336730 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.336572    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.336730 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.336578    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.337757 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.337742    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.337801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.337773    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 19:16:25.338465 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.338452    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 19:16:25.338538 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.338475    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 19:16:25.338538 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.338485    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeHasSufficientPID"
Jul 23 19:16:25.342280 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.342261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87ee8d79da3abe7938739ebd27b4b8c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal\" (UID: \"87ee8d79da3abe7938739ebd27b4b8c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.342363 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.342291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87ee8d79da3abe7938739ebd27b4b8c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal\" (UID: \"87ee8d79da3abe7938739ebd27b4b8c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.342363 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.342315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd1baf3c99400a5595a21fea2397e02-config\") pod \"kube-apiserver-proxy-ip-10-0-140-14.ec2.internal\" (UID: \"dbd1baf3c99400a5595a21fea2397e02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.344949 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.344934    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.357247 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.357227    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-14.ec2.internal\" not found" node="ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.360732 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.360717    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-14.ec2.internal\" not found" node="ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.442626 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.442589    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87ee8d79da3abe7938739ebd27b4b8c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal\" (UID: \"87ee8d79da3abe7938739ebd27b4b8c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.442626 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.442625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87ee8d79da3abe7938739ebd27b4b8c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal\" (UID: \"87ee8d79da3abe7938739ebd27b4b8c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.442872 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.442649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd1baf3c99400a5595a21fea2397e02-config\") pod \"kube-apiserver-proxy-ip-10-0-140-14.ec2.internal\" (UID: \"dbd1baf3c99400a5595a21fea2397e02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.442872 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.442732    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd1baf3c99400a5595a21fea2397e02-config\") pod \"kube-apiserver-proxy-ip-10-0-140-14.ec2.internal\" (UID: \"dbd1baf3c99400a5595a21fea2397e02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.442872 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.442743    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87ee8d79da3abe7938739ebd27b4b8c6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal\" (UID: \"87ee8d79da3abe7938739ebd27b4b8c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.442872 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.442783    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87ee8d79da3abe7938739ebd27b4b8c6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal\" (UID: \"87ee8d79da3abe7938739ebd27b4b8c6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.445715 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.445700    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.546572 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.546490    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.647085 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.647045    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.661239 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.661212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.664212 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.664194    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:25.747437 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.747397    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.848034 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:25.847963    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-14.ec2.internal\" not found"
Jul 23 19:16:25.944134 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.944107    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:16:25.983410 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.983378    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 19:16:25.983860 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.983546    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 19:16:25.983860 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:25.983571    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 19:16:26.019750 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.019716    2566 apiserver.go:52] "Watching apiserver"
Jul 23 19:16:26.039313 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.039280    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 19:16:26.039608 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.039592    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:26.062428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.061154    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 19:16:26.062428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.062326    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-7wsbr","openshift-multus/network-metrics-daemon-qbd7m","openshift-network-operator/iptables-alerter-5f8hj","openshift-ovn-kubernetes/ovnkube-node-c5mx6","kube-system/global-pull-secret-syncer-5km4c","kube-system/konnectivity-agent-ztdvt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm","openshift-dns/node-resolver-l8vrb","openshift-image-registry/node-ca-89stp","openshift-multus/multus-additional-cni-plugins-xd8qc","openshift-network-diagnostics/network-check-target-69v2n","openshift-cluster-node-tuning-operator/tuned-lpzk5"]
Jul 23 19:16:26.064535 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.064515    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.065886 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.065865    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.065972 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.065937    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:26.067022 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.067001    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.067213 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.067189    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 19:16:26.067322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.067259    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r4brz\""
Jul 23 19:16:26.067588 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.067571    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.068453 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.068435    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.069712 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.069697    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.070990 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.070976    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.071935 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.071913    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.072361 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.072347    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.072755 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.072736    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wmh8z\""
Jul 23 19:16:26.072878 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.072861    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 19:16:26.072945 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.072862    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 19:16:26.072945 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.072893    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.073063 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.073049    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 19:16:26.073110 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.073053    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.073946 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.073932    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.074971 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.074950    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 19:16:26.075052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.074950    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 19:11:25 +0000 UTC" deadline="2028-02-17 07:59:49.27501236 +0000 UTC"
Jul 23 19:16:26.075052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075010    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13764h43m23.200007226s"
Jul 23 19:16:26.075244 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075223    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dl4vv\""
Jul 23 19:16:26.075388 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075306    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 19:16:26.075487 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075307    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tqrpd\""
Jul 23 19:16:26.075577 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075560    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 19:16:26.075577 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075568    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 19:16:26.075680 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075587    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:26.075680 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075624    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7pztv\""
Jul 23 19:16:26.075680 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075638    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"
Jul 23 19:16:26.075680 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075649    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.075680 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075661    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.075882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075733    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 19:16:26.075882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075772    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 19:16:26.075882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075849    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.075882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075866    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.075999 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075867    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9hnvp\""
Jul 23 19:16:26.075999 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.075906    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.076059 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.076014    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.076091 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.076049    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:26.076124 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.076089    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 19:16:26.076985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.076969    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.079194 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.078829    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.079194 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.078889    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:26.080170 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.080151    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal"]
Jul 23 19:16:26.080270 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.080253    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.082188 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.082170    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 19:16:26.088416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.088401    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 19:16:26.088781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.088762    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kg2sw\""
Jul 23 19:16:26.093010 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.092989    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.093135 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.093104    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 19:16:26.093135 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.093118    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 19:16:26.093222 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.093104    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fp2r2\""
Jul 23 19:16:26.093265 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.093223    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mw24f\""
Jul 23 19:16:26.093299 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.093270    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.097049 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.097030    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 19:16:26.105203 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.105186    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 19:16:26.128641 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.128615    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal"]
Jul 23 19:16:26.128738 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.128646    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 19:16:26.141399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.141376    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 19:16:26.145663 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145643    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-cnibin\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.145762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145669    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-run\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.145762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66skl\" (UniqueName: \"kubernetes.io/projected/23dc984a-2a51-401e-822c-1a67fe34da8e-kube-api-access-66skl\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.145762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145719    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-log-socket\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.145762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-cni-netd\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.145762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145752    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-etc-selinux\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.145985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-cni-bin\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.145985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.145985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-ovnkube-config\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.145985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-hosts-file\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.145985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145915    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-tmp-dir\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.145985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145969    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-os-release\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.145997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-host\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146014    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-systemd-units\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146061    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-etc-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146086    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-node-log\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ace550c2-7d09-4222-a037-f6be6a6372e0-host-slash\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-systemd\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146171    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7360a644-2bb7-40d4-9231-765a6df2839f-ovn-node-metrics-cert\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146207    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-socket-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.146238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146238    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g924\" (UniqueName: \"kubernetes.io/projected/3db3290f-1a19-427b-95fa-44be99fb7ea6-kube-api-access-7g924\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffcwv\" (UniqueName: \"kubernetes.io/projected/ace550c2-7d09-4222-a037-f6be6a6372e0-kube-api-access-ffcwv\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146316    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-kubelet\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-var-lib-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146363    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttfgb\" (UniqueName: \"kubernetes.io/projected/7360a644-2bb7-40d4-9231-765a6df2839f-kube-api-access-ttfgb\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-serviceca\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d548d642-8e95-4d45-a44a-5cbba4f7565a-konnectivity-ca\") pod \"konnectivity-agent-ztdvt\" (UID: \"d548d642-8e95-4d45-a44a-5cbba4f7565a\") " pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysctl-conf\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146457    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc3608de-9ecd-4b56-a168-e330b2a877d6-tmp\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-ovn\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146523    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-modprobe-d\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146546    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-lib-modules\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.146576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/514184d7-e116-4ec4-a5b9-b2384ddf0468-kubelet-config\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146602    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146634    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ace550c2-7d09-4222-a037-f6be6a6372e0-iptables-alerter-script\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-systemd\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-conf-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-kubelet\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146745    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2qgv\" (UniqueName: \"kubernetes.io/projected/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-kube-api-access-h2qgv\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146774    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysconfig\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146804    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-system-cni-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146824    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-multus-certs\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146847    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-host\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146867    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysctl-d\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5e3fcc9-f961-4563-9f75-7db839edf89a-cni-binary-copy\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-slash\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.147106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146951    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146965    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146980    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.146995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-tuned\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-cni-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-k8s-cni-cncf-io\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-ovnkube-script-lib\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkqc6\" (UniqueName: \"kubernetes.io/projected/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-kube-api-access-nkqc6\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147074    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d548d642-8e95-4d45-a44a-5cbba4f7565a-agent-certs\") pod \"konnectivity-agent-ztdvt\" (UID: \"d548d642-8e95-4d45-a44a-5cbba4f7565a\") " pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147093    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnpsp\" (UniqueName: \"kubernetes.io/projected/1dedd804-d18a-41d0-a270-642769947be6-kube-api-access-qnpsp\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-sys\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-os-release\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-socket-dir-parent\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-run-netns\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147199    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-sys-fs\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.147824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44v2v\" (UniqueName: \"kubernetes.io/projected/bc3608de-9ecd-4b56-a168-e330b2a877d6-kube-api-access-44v2v\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-cnibin\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-netns\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-cni-multus\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147324    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-hostroot\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-etc-kubernetes\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147393    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-cni-bin\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-device-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147454    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-daemon-config\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f4hl\" (UniqueName: \"kubernetes.io/projected/c5e3fcc9-f961-4563-9f75-7db839edf89a-kube-api-access-7f4hl\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-run-ovn-kubernetes\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147545    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-env-overrides\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-registration-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147606    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-kubernetes\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-var-lib-kubelet\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.148287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147718    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/514184d7-e116-4ec4-a5b9-b2384ddf0468-dbus\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.148811 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.147744    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-system-cni-dir\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.150740 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.150724    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-58s8c"
Jul 23 19:16:26.170709 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.170673    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-58s8c"
Jul 23 19:16:26.238338 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.238308    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:16:26.248099 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248062    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-var-lib-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttfgb\" (UniqueName: \"kubernetes.io/projected/7360a644-2bb7-40d4-9231-765a6df2839f-kube-api-access-ttfgb\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-serviceca\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.248163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248141    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d548d642-8e95-4d45-a44a-5cbba4f7565a-konnectivity-ca\") pod \"konnectivity-agent-ztdvt\" (UID: \"d548d642-8e95-4d45-a44a-5cbba4f7565a\") " pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.248163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysctl-conf\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc3608de-9ecd-4b56-a168-e330b2a877d6-tmp\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248184    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-var-lib-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-ovn\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-ovn\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248245    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-modprobe-d\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248280    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-lib-modules\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248333 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/514184d7-e116-4ec4-a5b9-b2384ddf0468-kubelet-config\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248362    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-modprobe-d\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ace550c2-7d09-4222-a037-f6be6a6372e0-iptables-alerter-script\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248367    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysctl-conf\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/514184d7-e116-4ec4-a5b9-b2384ddf0468-kubelet-config\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-systemd\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248510    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-lib-modules\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248508    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-conf-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-kubelet\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248566    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-systemd\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248573    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-conf-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2qgv\" (UniqueName: \"kubernetes.io/projected/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-kube-api-access-h2qgv\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-kubelet\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248618    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 19:16:26.248687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-serviceca\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysconfig\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-system-cni-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-multus-certs\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-host\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248858    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d548d642-8e95-4d45-a44a-5cbba4f7565a-konnectivity-ca\") pod \"konnectivity-agent-ztdvt\" (UID: \"d548d642-8e95-4d45-a44a-5cbba4f7565a\") " pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysconfig\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysctl-d\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-system-cni-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248913    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-multus-certs\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248949    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-host\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ace550c2-7d09-4222-a037-f6be6a6372e0-iptables-alerter-script\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.248981    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5e3fcc9-f961-4563-9f75-7db839edf89a-cni-binary-copy\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-slash\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249060    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-sysctl-d\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249067    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.249542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249087    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-tuned\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249198    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-cni-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249203    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-slash\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249218    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-k8s-cni-cncf-io\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249262    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-k8s-cni-cncf-io\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-ovnkube-script-lib\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkqc6\" (UniqueName: \"kubernetes.io/projected/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-kube-api-access-nkqc6\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d548d642-8e95-4d45-a44a-5cbba4f7565a-agent-certs\") pod \"konnectivity-agent-ztdvt\" (UID: \"d548d642-8e95-4d45-a44a-5cbba4f7565a\") " pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-cni-dir\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249612    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5e3fcc9-f961-4563-9f75-7db839edf89a-cni-binary-copy\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249623    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnpsp\" (UniqueName: \"kubernetes.io/projected/1dedd804-d18a-41d0-a270-642769947be6-kube-api-access-qnpsp\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249658    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-sys\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249685    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-os-release\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.250428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249764    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-socket-dir-parent\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249792    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-run-netns\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249808    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-sys\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-sys-fs\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44v2v\" (UniqueName: \"kubernetes.io/projected/bc3608de-9ecd-4b56-a168-e330b2a877d6-kube-api-access-44v2v\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249882    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-os-release\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-cnibin\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.249916    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249933    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-sys-fs\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-run-netns\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.249982    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-netns\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.250003    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:26.749979077 +0000 UTC m=+2.038578651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250038    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-cnibin\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250050    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-run-netns\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-cni-multus\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.251498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250105    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-socket-dir-parent\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-hostroot\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-etc-kubernetes\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-cni-multus\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-cni-bin\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250222    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-device-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250247    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-hostroot\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-daemon-config\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250248    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-etc-kubernetes\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-ovnkube-script-lib\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250359    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-device-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250342    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f4hl\" (UniqueName: \"kubernetes.io/projected/c5e3fcc9-f961-4563-9f75-7db839edf89a-kube-api-access-7f4hl\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-run-ovn-kubernetes\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-cni-bin\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250546    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-env-overrides\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-run-ovn-kubernetes\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250607    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-registration-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.252391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-kubernetes\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250716    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5e3fcc9-f961-4563-9f75-7db839edf89a-multus-daemon-config\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250723    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-registration-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250732    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-var-lib-kubelet\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/514184d7-e116-4ec4-a5b9-b2384ddf0468-dbus\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-var-lib-kubelet\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250795    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-system-cni-dir\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-kubernetes\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250837    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/514184d7-e116-4ec4-a5b9-b2384ddf0468-dbus\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250853    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-cnibin\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250821    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-cnibin\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250895    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-run\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66skl\" (UniqueName: \"kubernetes.io/projected/23dc984a-2a51-401e-822c-1a67fe34da8e-kube-api-access-66skl\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-system-cni-dir\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250958    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-log-socket\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-run\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.253845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-cni-netd\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-host-cni-netd\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251041    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-log-socket\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.250977    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-env-overrides\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251190    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1dedd804-d18a-41d0-a270-642769947be6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251626    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-etc-selinux\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-cni-bin\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251709    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251737    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-ovnkube-config\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-etc-selinux\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251763    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-hosts-file\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-tmp-dir\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251789    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251760    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-cni-bin\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251819    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-os-release\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251844    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-host\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251869    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-systemd-units\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.254733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-etc-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-node-log\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.251973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ace550c2-7d09-4222-a037-f6be6a6372e0-host-slash\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-systemd\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7360a644-2bb7-40d4-9231-765a6df2839f-ovn-node-metrics-cert\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252051    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-socket-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252081    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g924\" (UniqueName: \"kubernetes.io/projected/3db3290f-1a19-427b-95fa-44be99fb7ea6-kube-api-access-7g924\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-tmp-dir\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-etc-openvswitch\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffcwv\" (UniqueName: \"kubernetes.io/projected/ace550c2-7d09-4222-a037-f6be6a6372e0-kube-api-access-ffcwv\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252206    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dedd804-d18a-41d0-a270-642769947be6-os-release\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-node-log\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc3608de-9ecd-4b56-a168-e330b2a877d6-host\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252342    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3db3290f-1a19-427b-95fa-44be99fb7ea6-socket-dir\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252416    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-run-systemd\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252501    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7360a644-2bb7-40d4-9231-765a6df2839f-ovnkube-config\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.255615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252513    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-kubelet\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5e3fcc9-f961-4563-9f75-7db839edf89a-host-var-lib-kubelet\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.252853    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.253110    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:16:26.753090618 +0000 UTC m=+2.041690213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.253106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7360a644-2bb7-40d4-9231-765a6df2839f-systemd-units\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ace550c2-7d09-4222-a037-f6be6a6372e0-host-slash\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.252029    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-hosts-file\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.253155    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bc3608de-9ecd-4b56-a168-e330b2a877d6-etc-tuned\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.253348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc3608de-9ecd-4b56-a168-e330b2a877d6-tmp\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.253622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d548d642-8e95-4d45-a44a-5cbba4f7565a-agent-certs\") pod \"konnectivity-agent-ztdvt\" (UID: \"d548d642-8e95-4d45-a44a-5cbba4f7565a\") " pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.256504 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.254532    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7360a644-2bb7-40d4-9231-765a6df2839f-ovn-node-metrics-cert\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.261736 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.261717    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:26.261849 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.261741    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:26.261849 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.261756    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:26.261849 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.261817    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:26.761802665 +0000 UTC m=+2.050402254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:26.264200 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.264161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttfgb\" (UniqueName: \"kubernetes.io/projected/7360a644-2bb7-40d4-9231-765a6df2839f-kube-api-access-ttfgb\") pod \"ovnkube-node-c5mx6\" (UID: \"7360a644-2bb7-40d4-9231-765a6df2839f\") " pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.265128 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.265100    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2qgv\" (UniqueName: \"kubernetes.io/projected/8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25-kube-api-access-h2qgv\") pod \"node-ca-89stp\" (UID: \"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25\") " pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.271238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.271194    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44v2v\" (UniqueName: \"kubernetes.io/projected/bc3608de-9ecd-4b56-a168-e330b2a877d6-kube-api-access-44v2v\") pod \"tuned-lpzk5\" (UID: \"bc3608de-9ecd-4b56-a168-e330b2a877d6\") " pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.271430 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.271257    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffcwv\" (UniqueName: \"kubernetes.io/projected/ace550c2-7d09-4222-a037-f6be6a6372e0-kube-api-access-ffcwv\") pod \"iptables-alerter-5f8hj\" (UID: \"ace550c2-7d09-4222-a037-f6be6a6372e0\") " pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.272096 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.272072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkqc6\" (UniqueName: \"kubernetes.io/projected/d4124a71-f788-4b4a-9ac7-388e2eeb2ec7-kube-api-access-nkqc6\") pod \"node-resolver-l8vrb\" (UID: \"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7\") " pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.272383 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.272365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnpsp\" (UniqueName: \"kubernetes.io/projected/1dedd804-d18a-41d0-a270-642769947be6-kube-api-access-qnpsp\") pod \"multus-additional-cni-plugins-xd8qc\" (UID: \"1dedd804-d18a-41d0-a270-642769947be6\") " pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.272580 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.272557    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66skl\" (UniqueName: \"kubernetes.io/projected/23dc984a-2a51-401e-822c-1a67fe34da8e-kube-api-access-66skl\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.273930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.273908    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f4hl\" (UniqueName: \"kubernetes.io/projected/c5e3fcc9-f961-4563-9f75-7db839edf89a-kube-api-access-7f4hl\") pod \"multus-7wsbr\" (UID: \"c5e3fcc9-f961-4563-9f75-7db839edf89a\") " pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.274024 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.273961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g924\" (UniqueName: \"kubernetes.io/projected/3db3290f-1a19-427b-95fa-44be99fb7ea6-kube-api-access-7g924\") pod \"aws-ebs-csi-driver-node-5p6hm\" (UID: \"3db3290f-1a19-427b-95fa-44be99fb7ea6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.385611 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.385538    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7wsbr"
Jul 23 19:16:26.389085 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.389062    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:16:26.391921 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.391900    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5f8hj"
Jul 23 19:16:26.401202 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.401171    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e3fcc9_f961_4563_9f75_7db839edf89a.slice/crio-2f601fe82b544be80fff36e6ad6ad101289af2214d47236680b382b5dfadc811 WatchSource:0}: Error finding container 2f601fe82b544be80fff36e6ad6ad101289af2214d47236680b382b5dfadc811: Status 404 returned error can't find the container with id 2f601fe82b544be80fff36e6ad6ad101289af2214d47236680b382b5dfadc811
Jul 23 19:16:26.401946 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.401923    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ee8d79da3abe7938739ebd27b4b8c6.slice/crio-cf2177b63ee9c7ae3366fcb823e70ee8475bdd6b7ac020e584cad5e9c9a0cc5b WatchSource:0}: Error finding container cf2177b63ee9c7ae3366fcb823e70ee8475bdd6b7ac020e584cad5e9c9a0cc5b: Status 404 returned error can't find the container with id cf2177b63ee9c7ae3366fcb823e70ee8475bdd6b7ac020e584cad5e9c9a0cc5b
Jul 23 19:16:26.402408 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.402389    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd1baf3c99400a5595a21fea2397e02.slice/crio-c0d35bb1a85505b58e1e96654f9a66023ee3c51bbc78ab72d2d0091571e717b8 WatchSource:0}: Error finding container c0d35bb1a85505b58e1e96654f9a66023ee3c51bbc78ab72d2d0091571e717b8: Status 404 returned error can't find the container with id c0d35bb1a85505b58e1e96654f9a66023ee3c51bbc78ab72d2d0091571e717b8
Jul 23 19:16:26.404769 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.404651    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace550c2_7d09_4222_a037_f6be6a6372e0.slice/crio-f1509febd55a0b66072632ae3c239b1a75ed1715807facc332267dad5e7c1328 WatchSource:0}: Error finding container f1509febd55a0b66072632ae3c239b1a75ed1715807facc332267dad5e7c1328: Status 404 returned error can't find the container with id f1509febd55a0b66072632ae3c239b1a75ed1715807facc332267dad5e7c1328
Jul 23 19:16:26.407208 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.407194    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:16:26.426344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.426283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:26.428497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.428470    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:26.435078 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.435053    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7360a644_2bb7_40d4_9231_765a6df2839f.slice/crio-3a3e3872fc77ee2ab1f6994637dba9e27ad57a4558b7f5ed65c28dea5bd1a8c1 WatchSource:0}: Error finding container 3a3e3872fc77ee2ab1f6994637dba9e27ad57a4558b7f5ed65c28dea5bd1a8c1: Status 404 returned error can't find the container with id 3a3e3872fc77ee2ab1f6994637dba9e27ad57a4558b7f5ed65c28dea5bd1a8c1
Jul 23 19:16:26.437792 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.437767    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd548d642_8e95_4d45_a44a_5cbba4f7565a.slice/crio-2ec60b723316d56240d7aaa1ddcce67c72f5a5a24098ec1899e5017dce5a6d15 WatchSource:0}: Error finding container 2ec60b723316d56240d7aaa1ddcce67c72f5a5a24098ec1899e5017dce5a6d15: Status 404 returned error can't find the container with id 2ec60b723316d56240d7aaa1ddcce67c72f5a5a24098ec1899e5017dce5a6d15
Jul 23 19:16:26.461532 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.461509    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8vrb"
Jul 23 19:16:26.466897 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.466875    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-89stp"
Jul 23 19:16:26.470132 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.470111    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4124a71_f788_4b4a_9ac7_388e2eeb2ec7.slice/crio-f6abd263c22240f726d6c92f4567dd06b368cd0b5713fdd19cd72de8e0295914 WatchSource:0}: Error finding container f6abd263c22240f726d6c92f4567dd06b368cd0b5713fdd19cd72de8e0295914: Status 404 returned error can't find the container with id f6abd263c22240f726d6c92f4567dd06b368cd0b5713fdd19cd72de8e0295914
Jul 23 19:16:26.475736 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.475712    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5c1b6f_bc55_49e3_bd1b_bce9051c2e25.slice/crio-743ad4b15a0c912e9cad18171a794cfc6cdf4dc7b52c472a74538f0f3d53da2a WatchSource:0}: Error finding container 743ad4b15a0c912e9cad18171a794cfc6cdf4dc7b52c472a74538f0f3d53da2a: Status 404 returned error can't find the container with id 743ad4b15a0c912e9cad18171a794cfc6cdf4dc7b52c472a74538f0f3d53da2a
Jul 23 19:16:26.481909 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.481890    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xd8qc"
Jul 23 19:16:26.489982 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.489960    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dedd804_d18a_41d0_a270_642769947be6.slice/crio-f5e8aeb63f9b4b97101fed7824ce42efd4007d76a1aea6fc581ace22e1d40627 WatchSource:0}: Error finding container f5e8aeb63f9b4b97101fed7824ce42efd4007d76a1aea6fc581ace22e1d40627: Status 404 returned error can't find the container with id f5e8aeb63f9b4b97101fed7824ce42efd4007d76a1aea6fc581ace22e1d40627
Jul 23 19:16:26.501941 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.501920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lpzk5"
Jul 23 19:16:26.507664 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.507644    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm"
Jul 23 19:16:26.511553 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.511531    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3608de_9ecd_4b56_a168_e330b2a877d6.slice/crio-a5bdf70bc2b2ba6af0ab75bf331ab6726aed0938939d682ef5437792a3824c78 WatchSource:0}: Error finding container a5bdf70bc2b2ba6af0ab75bf331ab6726aed0938939d682ef5437792a3824c78: Status 404 returned error can't find the container with id a5bdf70bc2b2ba6af0ab75bf331ab6726aed0938939d682ef5437792a3824c78
Jul 23 19:16:26.516442 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:16:26.516417    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db3290f_1a19_427b_95fa_44be99fb7ea6.slice/crio-784965491e2d50e2b8d906efc1b0dfbb5bdce9229664b41e7eaab8726c94b9c3 WatchSource:0}: Error finding container 784965491e2d50e2b8d906efc1b0dfbb5bdce9229664b41e7eaab8726c94b9c3: Status 404 returned error can't find the container with id 784965491e2d50e2b8d906efc1b0dfbb5bdce9229664b41e7eaab8726c94b9c3
Jul 23 19:16:26.518260 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.518238    2566 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-5p6hm_openshift-cluster-csi-drivers(3db3290f-1a19-427b-95fa-44be99fb7ea6): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 23 19:16:26.519008 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.518982    2566 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-5p6hm_openshift-cluster-csi-drivers(3db3290f-1a19-427b-95fa-44be99fb7ea6): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 23 19:16:26.520279 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.520260    2566 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-5p6hm_openshift-cluster-csi-drivers(3db3290f-1a19-427b-95fa-44be99fb7ea6): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 23 19:16:26.520358 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.520312    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" podUID="3db3290f-1a19-427b-95fa-44be99fb7ea6"
Jul 23 19:16:26.756841 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.756800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:26.757026 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.756859    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:26.757026 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.756954    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:26.757026 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.756958    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:26.757026 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.757014    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:16:27.756998653 +0000 UTC m=+3.045598224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:26.757026 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.757028    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:27.75702133 +0000 UTC m=+3.045620901 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:26.834534 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.834490    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:16:26.858029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:26.857985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:26.858221 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.858164    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:26.858221 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.858185    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:26.858221 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.858196    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:26.858324 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:26.858250    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:27.858235059 +0000 UTC m=+3.146834634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:27.173599 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.173522    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 19:11:26 +0000 UTC" deadline="2028-01-06 21:51:36.892274168 +0000 UTC"
Jul 23 19:16:27.173599 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.173598    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12770h35m9.718680652s"
Jul 23 19:16:27.232937 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.232904    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:27.233127 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.233069    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:27.252673 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.252544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal" event={"ID":"87ee8d79da3abe7938739ebd27b4b8c6","Type":"ContainerStarted","Data":"cf2177b63ee9c7ae3366fcb823e70ee8475bdd6b7ac020e584cad5e9c9a0cc5b"}
Jul 23 19:16:27.261041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.260993    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lpzk5" event={"ID":"bc3608de-9ecd-4b56-a168-e330b2a877d6","Type":"ContainerStarted","Data":"a5bdf70bc2b2ba6af0ab75bf331ab6726aed0938939d682ef5437792a3824c78"}
Jul 23 19:16:27.269138 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.269102    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerStarted","Data":"f5e8aeb63f9b4b97101fed7824ce42efd4007d76a1aea6fc581ace22e1d40627"}
Jul 23 19:16:27.284681 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.284638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-89stp" event={"ID":"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25","Type":"ContainerStarted","Data":"743ad4b15a0c912e9cad18171a794cfc6cdf4dc7b52c472a74538f0f3d53da2a"}
Jul 23 19:16:27.294366 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.294326    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"3a3e3872fc77ee2ab1f6994637dba9e27ad57a4558b7f5ed65c28dea5bd1a8c1"}
Jul 23 19:16:27.305084 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.305044    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" event={"ID":"3db3290f-1a19-427b-95fa-44be99fb7ea6","Type":"ContainerStarted","Data":"784965491e2d50e2b8d906efc1b0dfbb5bdce9229664b41e7eaab8726c94b9c3"}
Jul 23 19:16:27.311264 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.311232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8vrb" event={"ID":"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7","Type":"ContainerStarted","Data":"f6abd263c22240f726d6c92f4567dd06b368cd0b5713fdd19cd72de8e0295914"}
Jul 23 19:16:27.315121 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.315073    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b13e6eb321323c63d8808dd2edd629ec9e6bb50a1f51642a22e1192625d06647\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a241c7873dcd9a33c18e22c17fc2266c2eba1ed07e551f8eb9e60ee375ef576a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0409c7725f34b67762c9af07735a2803f1edbba13924022918025dff1863c18c\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" podUID="3db3290f-1a19-427b-95fa-44be99fb7ea6"
Jul 23 19:16:27.317091 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.317062    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ztdvt" event={"ID":"d548d642-8e95-4d45-a44a-5cbba4f7565a","Type":"ContainerStarted","Data":"2ec60b723316d56240d7aaa1ddcce67c72f5a5a24098ec1899e5017dce5a6d15"}
Jul 23 19:16:27.328462 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.328421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5f8hj" event={"ID":"ace550c2-7d09-4222-a037-f6be6a6372e0","Type":"ContainerStarted","Data":"f1509febd55a0b66072632ae3c239b1a75ed1715807facc332267dad5e7c1328"}
Jul 23 19:16:27.339070 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.339036    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7wsbr" event={"ID":"c5e3fcc9-f961-4563-9f75-7db839edf89a","Type":"ContainerStarted","Data":"2f601fe82b544be80fff36e6ad6ad101289af2214d47236680b382b5dfadc811"}
Jul 23 19:16:27.347405 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.347359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal" event={"ID":"dbd1baf3c99400a5595a21fea2397e02","Type":"ContainerStarted","Data":"c0d35bb1a85505b58e1e96654f9a66023ee3c51bbc78ab72d2d0091571e717b8"}
Jul 23 19:16:27.765497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.765405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:27.765497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.765482    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:27.765744 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.765617    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:27.765744 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.765680    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:16:29.765662359 +0000 UTC m=+5.054261932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:27.766121 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.766098    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:27.766193 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.766160    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:29.766144227 +0000 UTC m=+5.054743801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:27.866523 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:27.866429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:27.866731 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.866641    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:27.866731 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.866666    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:27.866731 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.866680    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:27.866939 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:27.866766    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:29.866744178 +0000 UTC m=+5.155343767 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:28.174042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:28.173908    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 19:11:26 +0000 UTC" deadline="2028-03-20 01:06:00.108255929 +0000 UTC"
Jul 23 19:16:28.174042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:28.173977    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14525h49m31.93428511s"
Jul 23 19:16:28.233118 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:28.233021    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:28.233118 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:28.233071    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:28.233412 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:28.233174    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:28.233412 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:28.233271    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:28.862196 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:28.862163    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 19:16:29.235240 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:29.235201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:29.235766 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.235337    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:29.784099 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:29.783087    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:29.784099 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:29.783161    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:29.784099 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.783309    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:29.784099 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.783372    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:16:33.783353974 +0000 UTC m=+9.071953564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:29.784099 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.783864    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:29.784099 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.783944    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:33.783921323 +0000 UTC m=+9.072520895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:29.883965 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:29.883919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:29.884162 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.884065    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:29.884162 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.884082    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:29.884162 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.884091    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:29.884162 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:29.884145    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:33.884130907 +0000 UTC m=+9.172730477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:30.233702 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:30.232935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:30.233702 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:30.233075    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:30.233702 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:30.233496    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:30.233702 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:30.233621    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:31.233814 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:31.233221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:31.233814 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:31.233422    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:32.233375 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:32.233329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:32.233551 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:32.233471    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:32.233929 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:32.233901    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:32.234314 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:32.234007    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:33.234073 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:33.234041    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:33.234506 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.234179    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:33.821378 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:33.821339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:33.821539 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:33.821410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:33.821584 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.821544    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:33.821615 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.821607    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:16:41.821588141 +0000 UTC m=+17.110187710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:33.822033 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.822013    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:33.822111 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.822094    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:41.822054849 +0000 UTC m=+17.110654425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:33.922815 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:33.922729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:33.923007 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.922934    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:33.923007 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.922974    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:33.923007 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.922988    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:33.923176 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:33.923046    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:41.923026904 +0000 UTC m=+17.211626481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:34.232952 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:34.232919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:34.233117 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:34.232919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:34.233117 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:34.233081    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:34.233235 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:34.233147    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:35.233750 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:35.233712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:35.234196 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:35.233850    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:36.233372 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:36.232880    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:36.233372 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:36.232918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:36.233372 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:36.233035    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:36.233372 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:36.233190    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:37.233258 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:37.233225    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:37.233658 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:37.233360    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:38.233434 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:38.233385    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:38.233434 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:38.233410    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:38.233889 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:38.233538    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:38.233889 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:38.233662    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:39.232824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:39.232790    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:39.233004 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:39.232929    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:40.232616 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:40.232567    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:40.233075 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:40.232750    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:40.233075 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:40.232781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:40.233075 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:40.232884    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:41.232659 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:41.232393    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:41.233077 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.232775    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:41.883949 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:41.883906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:41.884120 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:41.883988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:41.884120 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.884072    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:41.884120 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.884087    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:41.884248 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.884137    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:16:57.884122396 +0000 UTC m=+33.172721964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:41.884248 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.884155    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:57.884146381 +0000 UTC m=+33.172745950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:41.984638 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:41.984600    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:41.984810 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.984792    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:41.984860 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.984816    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:41.984860 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.984826    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:41.984936 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:41.984879    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:16:57.984866827 +0000 UTC m=+33.273466395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:42.232950 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:42.232919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:42.233416 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:42.233043    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:42.233416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:42.233092    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:42.233416 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:42.233172    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:43.233206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:43.233174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:43.233630 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:43.233280    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:44.233045 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:44.232994    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:44.233045 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:44.233029    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:44.233588 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:44.233113    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:44.233588 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:44.233255    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:45.233112 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:45.233079    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:45.233303 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:45.233201    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:46.233278 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:46.233207    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:46.233396 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:46.233221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:46.233396 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:46.233335    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:46.233396 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:46.233375    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:47.232898 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.232729    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:47.233014 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:47.232973    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:47.404305 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.404271    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ztdvt" event={"ID":"d548d642-8e95-4d45-a44a-5cbba4f7565a","Type":"ContainerStarted","Data":"81c9c567cc5f08a520b918e42f7b732f511edc249c0405c7cd1683e6f270ffb2"}
Jul 23 19:16:47.406125 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.406093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7wsbr" event={"ID":"c5e3fcc9-f961-4563-9f75-7db839edf89a","Type":"ContainerStarted","Data":"11c074b881acb5a4e4c423c69a73b042f4a0ef6c11c18f3429a73e3dadc7b26c"}
Jul 23 19:16:47.408106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.408049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal" event={"ID":"dbd1baf3c99400a5595a21fea2397e02","Type":"ContainerStarted","Data":"133f207ee49a6331a2a022bf0e564ef86f7c72636f3513ed04234f279a1b301c"}
Jul 23 19:16:47.409776 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.409740    2566 generic.go:356] "Generic (PLEG): container finished" podID="87ee8d79da3abe7938739ebd27b4b8c6" containerID="cc9ebd997e72911f607239eb47eddf9dab623be4f4c61613bfa6accb09cf599c" exitCode=0
Jul 23 19:16:47.409872 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.409822    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal" event={"ID":"87ee8d79da3abe7938739ebd27b4b8c6","Type":"ContainerDied","Data":"cc9ebd997e72911f607239eb47eddf9dab623be4f4c61613bfa6accb09cf599c"}
Jul 23 19:16:47.411550 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.411523    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lpzk5" event={"ID":"bc3608de-9ecd-4b56-a168-e330b2a877d6","Type":"ContainerStarted","Data":"da4a620a5d9d9e6d27b1309298a12c01247fde869cbce7accc09b5c8d6c75bfd"}
Jul 23 19:16:47.413368 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.413344    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dedd804-d18a-41d0-a270-642769947be6" containerID="de45c6fa3c02e880ff3a49f423309a595fbb87f0f1632e7cd76e0046024ac760" exitCode=0
Jul 23 19:16:47.413468 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.413369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerDied","Data":"de45c6fa3c02e880ff3a49f423309a595fbb87f0f1632e7cd76e0046024ac760"}
Jul 23 19:16:47.415338 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.415305    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-89stp" event={"ID":"8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25","Type":"ContainerStarted","Data":"3d9798076558ba43c0d44906014aecd47c3bc9e1760ade05dfa10e2e660bc611"}
Jul 23 19:16:47.418260 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.418240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"88a6e571fcf1346cc241d533509bbe8ffd8d666ae88360bc7fb964c8a4cdf5c5"}
Jul 23 19:16:47.418366 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.418313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"67295ab8b80aea93446bdfda93f989a1ad72564aa9d14864cc6e656af94b8b98"}
Jul 23 19:16:47.418425 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.418363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"30645d06103dbdf67b7b995973eec286d9f38ea17150c0d9c7147c2b16deebe6"}
Jul 23 19:16:47.418425 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.418414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"18c1a2c63ca7ec6ac0f6e9764916135c9f3570ddc3a2152e012e492311825e77"}
Jul 23 19:16:47.418503 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.418460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"f677f04c2dabb9e641413c24d91882545324975e0ab48c8022b9d012389b43e2"}
Jul 23 19:16:47.419834 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.419812    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8vrb" event={"ID":"d4124a71-f788-4b4a-9ac7-388e2eeb2ec7","Type":"ContainerStarted","Data":"c0af5e505adc85fc3c88ad2745e2385be1da65e17ea9e1b0daa996e62e64199b"}
Jul 23 19:16:47.440549 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.440493    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ztdvt" podStartSLOduration=2.495581693 podStartE2EDuration="22.440474352s" totalImagesPullingTime="19.944892659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:47.420824107 +0000 UTC m=+22.709423699" watchObservedRunningTime="2026-07-23 19:16:47.440474352 +0000 UTC m=+22.729073947"
Jul 23 19:16:47.440780 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.440742    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-89stp" podStartSLOduration=2.533483007 podStartE2EDuration="22.440731166s" totalImagesPullingTime="19.907248159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:47.440220754 +0000 UTC m=+22.728820347" watchObservedRunningTime="2026-07-23 19:16:47.440731166 +0000 UTC m=+22.729330765"
Jul 23 19:16:47.484032 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.483979    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lpzk5" podStartSLOduration=2.560137196 podStartE2EDuration="22.483963584s" totalImagesPullingTime="19.923826388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:47.483608765 +0000 UTC m=+22.772208357" watchObservedRunningTime="2026-07-23 19:16:47.483963584 +0000 UTC m=+22.772563175"
Jul 23 19:16:47.518879 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.518781    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-14.ec2.internal" podStartSLOduration=21.518763922 podStartE2EDuration="21.518763922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:47.518642671 +0000 UTC m=+22.807242264" watchObservedRunningTime="2026-07-23 19:16:47.518763922 +0000 UTC m=+22.807363514"
Jul 23 19:16:47.534523 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.534473    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l8vrb" podStartSLOduration=2.563428081 podStartE2EDuration="22.534454483s" totalImagesPullingTime="19.971026402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:47.534371313 +0000 UTC m=+22.822970904" watchObservedRunningTime="2026-07-23 19:16:47.534454483 +0000 UTC m=+22.823054077"
Jul 23 19:16:47.557439 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:47.557395    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7wsbr" podStartSLOduration=2.504262175 podStartE2EDuration="22.557380064s" totalImagesPullingTime="20.053117889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:47.557130491 +0000 UTC m=+22.845730085" watchObservedRunningTime="2026-07-23 19:16:47.557380064 +0000 UTC m=+22.845979704"
Jul 23 19:16:48.233210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.232890    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:48.233284 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.232951    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:48.233334 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:48.233218    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:48.233377 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:48.233353    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:48.423888 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.423849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5f8hj" event={"ID":"ace550c2-7d09-4222-a037-f6be6a6372e0","Type":"ContainerStarted","Data":"bc47a33b87846aaa450c96d8efd37fe04f295cb43051b8d33bb13bcf48edab94"}
Jul 23 19:16:48.425985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.425955    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal" event={"ID":"87ee8d79da3abe7938739ebd27b4b8c6","Type":"ContainerStarted","Data":"db20e7a4f430caa15959ae4e12343e1d23ef598329881e54d759f0804f456170"}
Jul 23 19:16:48.429349 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.429313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"19997ecf093b43a0be1597b7a990b21d074eab2484162fc4aa950e7e61d29043"}
Jul 23 19:16:48.431490 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.431044    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" event={"ID":"3db3290f-1a19-427b-95fa-44be99fb7ea6","Type":"ContainerStarted","Data":"2c9c26cafc1b58bb052954415e47b71970a8565e882b299646f9f5a4860a59db"}
Jul 23 19:16:48.444061 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:48.444011    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5f8hj" podStartSLOduration=3.413149167 podStartE2EDuration="23.443997379s" totalImagesPullingTime="20.030848212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:48.443971197 +0000 UTC m=+23.732570789" watchObservedRunningTime="2026-07-23 19:16:48.443997379 +0000 UTC m=+23.732596969"
Jul 23 19:16:49.232517 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:49.232490    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:49.232810 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:49.232609    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:49.520273 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:49.520245    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 19:16:50.112205 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.112106    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T19:16:49.520267289Z","UUID":"1cc55fed-9901-4a68-9399-23c58cc1834d","Handler":null,"Name":"","Endpoint":""}
Jul 23 19:16:50.123274 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.123247    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 19:16:50.123499 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.123301    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 19:16:50.232892 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.232871    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:50.232991 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.232888    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:50.233037 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:50.233004    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:50.233123 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:50.233105    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:50.441486 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.441448    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"3a01bf516b6b93ea0a425c5408f2dc829e7c6f9ed4cf7e07e518fa1fb0838ab2"}
Jul 23 19:16:50.443615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:50.443589    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" event={"ID":"3db3290f-1a19-427b-95fa-44be99fb7ea6","Type":"ContainerStarted","Data":"fc0299c9c8b280b77edc59bf51fa57de9be733656c4cd0ea99c77c7ba54542e4"}
Jul 23 19:16:51.232884 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:51.232842    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:51.233365 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:51.232962    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:51.499840 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:51.499171    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:51.500219 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:51.500186    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:51.517789 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:51.517727    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-14.ec2.internal" podStartSLOduration=25.517709752000002 podStartE2EDuration="25.517709752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:48.461351235 +0000 UTC m=+23.749950825" watchObservedRunningTime="2026-07-23 19:16:51.517709752 +0000 UTC m=+26.806309344"
Jul 23 19:16:52.233201 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.233167    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:52.233541 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:52.233277    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:52.233541 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.233330    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:52.233541 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:52.233417    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:52.451415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.451137    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" event={"ID":"3db3290f-1a19-427b-95fa-44be99fb7ea6","Type":"ContainerStarted","Data":"ba4ff71615dfb61983731e41dc0dba2abf6cc820a6abbed10ac3af86e67df1ee"}
Jul 23 19:16:52.453352 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.453224    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dedd804-d18a-41d0-a270-642769947be6" containerID="f1d25c9acafbb95df0b5bebcd22d753c3dad7491da6fe0c4db35e5eb6f7ed5e3" exitCode=0
Jul 23 19:16:52.453352 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.453338    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerDied","Data":"f1d25c9acafbb95df0b5bebcd22d753c3dad7491da6fe0c4db35e5eb6f7ed5e3"}
Jul 23 19:16:52.457958 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.457802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" event={"ID":"7360a644-2bb7-40d4-9231-765a6df2839f","Type":"ContainerStarted","Data":"dce60578a0bfb53ab98c43ac17253b3902fdffe08a315a2357f7010db1cdbe1a"}
Jul 23 19:16:52.458318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.458105    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:52.458318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.458133    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:52.458318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.458163    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:52.458912 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.458506    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ztdvt"
Jul 23 19:16:52.476712 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.476513    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:52.477440 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.477381    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:52.487847 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.487778    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5p6hm" podStartSLOduration=0.675440084 podStartE2EDuration="26.487765207s" totalImagesPullingTime="25.812325123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-23 19:16:52.469812572 +0000 UTC m=+27.758412165" watchObservedRunningTime="2026-07-23 19:16:52.487765207 +0000 UTC m=+27.776364797"
Jul 23 19:16:52.538207 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:52.538156    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6" podStartSLOduration=7.280655354 podStartE2EDuration="27.53814278s" totalImagesPullingTime="20.257487426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:16:52.537502557 +0000 UTC m=+27.826102148" watchObservedRunningTime="2026-07-23 19:16:52.53814278 +0000 UTC m=+27.826742370"
Jul 23 19:16:53.013909 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:53.013872    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:16:53.233306 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:53.233257    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:53.234236 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:53.233414    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:54.232715 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:54.232665    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:54.232929 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:54.232690    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:54.232929 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:54.232790    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:54.232929 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:54.232858    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:54.463992 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:54.463957    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dedd804-d18a-41d0-a270-642769947be6" containerID="13fa166e3759422452c9a69eef0003f0c61ef976db14c100b30876a51c27ba91" exitCode=0
Jul 23 19:16:54.464434 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:54.464052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerDied","Data":"13fa166e3759422452c9a69eef0003f0c61ef976db14c100b30876a51c27ba91"}
Jul 23 19:16:55.233649 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:55.233623    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:55.233820 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:55.233723    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:56.233402 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:56.233226    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:56.233836 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:56.233230    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:56.233836 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:56.233468    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:56.233836 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:56.233548    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:56.470137 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:56.470107    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dedd804-d18a-41d0-a270-642769947be6" containerID="ae109776b914ca63e03d392603ee54b698f95b96fde14f235784b133b6d06629" exitCode=0
Jul 23 19:16:56.470282 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:56.470156    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerDied","Data":"ae109776b914ca63e03d392603ee54b698f95b96fde14f235784b133b6d06629"}
Jul 23 19:16:57.233046 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:57.233011    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:57.233229 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:57.233196    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:16:57.907828 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:57.907789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:57.908303 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:57.907867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:57.908303 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:57.907967    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:57.908303 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:57.908007    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:57.908303 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:57.908048    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs podName:23dc984a-2a51-401e-822c-1a67fe34da8e nodeName:}" failed. No retries permitted until 2026-07-23 19:17:29.908026042 +0000 UTC m=+65.196625616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs") pod "network-metrics-daemon-qbd7m" (UID: "23dc984a-2a51-401e-822c-1a67fe34da8e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 19:16:57.908303 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:57.908066    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret podName:514184d7-e116-4ec4-a5b9-b2384ddf0468 nodeName:}" failed. No retries permitted until 2026-07-23 19:17:29.908059252 +0000 UTC m=+65.196658821 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret") pod "global-pull-secret-syncer-5km4c" (UID: "514184d7-e116-4ec4-a5b9-b2384ddf0468") : object "kube-system"/"original-pull-secret" not registered
Jul 23 19:16:58.008741 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:58.008700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:58.008923 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:58.008863    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 19:16:58.008923 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:58.008884    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 19:16:58.008923 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:58.008898    2566 projected.go:196] Error preparing data for projected volume kube-api-access-xw9nr for pod openshift-network-diagnostics/network-check-target-69v2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:58.009075 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:58.008949    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr podName:325ebcee-edbf-49ad-b9a0-ae16bffcb462 nodeName:}" failed. No retries permitted until 2026-07-23 19:17:30.008934541 +0000 UTC m=+65.297534113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw9nr" (UniqueName: "kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr") pod "network-check-target-69v2n" (UID: "325ebcee-edbf-49ad-b9a0-ae16bffcb462") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 19:16:58.232433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:58.232354    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:16:58.232433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:58.232383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:16:58.232613 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:58.232495    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:16:58.232695 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:58.232630    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:16:59.233054 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:16:59.233024    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:16:59.233468 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:16:59.233157    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:00.233094 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:00.233059    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:00.233548 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:00.233198    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:00.233548 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:00.233249    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:00.233548 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:00.233357    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:01.232565 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:01.232532    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:01.232786 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:01.232624    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:01.485303 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:01.485221    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dedd804-d18a-41d0-a270-642769947be6" containerID="d705960167a325042a267fb72c6c73bcae47ca0d8b2d44e3c46393c9877a833b" exitCode=0
Jul 23 19:17:01.485303 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:01.485272    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerDied","Data":"d705960167a325042a267fb72c6c73bcae47ca0d8b2d44e3c46393c9877a833b"}
Jul 23 19:17:02.233042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:02.232997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:02.233202 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:02.232997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:02.233202 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:02.233127    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:02.233202 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:02.233186    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:02.490619 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:02.490548    2566 generic.go:356] "Generic (PLEG): container finished" podID="1dedd804-d18a-41d0-a270-642769947be6" containerID="839868f712073f035c86c502b1e8f0dcce9b8868c1244700d4600145ad3fe131" exitCode=0
Jul 23 19:17:02.490619 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:02.490599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerDied","Data":"839868f712073f035c86c502b1e8f0dcce9b8868c1244700d4600145ad3fe131"}
Jul 23 19:17:03.233104 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:03.233070    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:03.233273 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:03.233173    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:03.497583 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:03.497498    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" event={"ID":"1dedd804-d18a-41d0-a270-642769947be6","Type":"ContainerStarted","Data":"330b6be6d2bcdbe1e0753c457e07707dadf6c4d8befdece30a20cc3bc59dbc0b"}
Jul 23 19:17:03.522133 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:03.522089    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xd8qc" podStartSLOduration=8.051180251 podStartE2EDuration="38.52207172s" totalImagesPullingTime="30.470891469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:03.521783798 +0000 UTC m=+38.810383388" watchObservedRunningTime="2026-07-23 19:17:03.52207172 +0000 UTC m=+38.810671310"
Jul 23 19:17:04.232690 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:04.232643    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:04.232879 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:04.232783    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:04.232879 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:04.232814    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:04.232997 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:04.232876    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:05.233507 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:05.233473    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:05.233943 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:05.233556    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:06.232947 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:06.232916    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:06.233108 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:06.232923    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:06.233108 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:06.233009    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:06.233108 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:06.233088    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:07.233277 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:07.233246    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:07.233622 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:07.233353    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:08.232612 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:08.232579    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:08.232612 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:08.232579    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:08.232874 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:08.232754    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:08.232874 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:08.232861    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:09.232637 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.232605    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:09.233007 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:09.232732    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:09.896632 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.896447    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5km4c"]
Jul 23 19:17:09.896873 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.896736    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:09.896873 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:09.896830    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:09.898156 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.898130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-69v2n"]
Jul 23 19:17:09.898306 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.898224    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:09.898381 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:09.898329    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:09.898864 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.898840    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qbd7m"]
Jul 23 19:17:09.898980 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:09.898938    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:09.899036 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:09.899020    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:11.233251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:11.233212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:11.233754 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:11.233324    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:12.232447 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:12.232411    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:12.232622 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:12.232416    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:12.232622 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:12.232510    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:12.232772 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:12.232610    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:13.233439 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:13.233409    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:13.233850 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:13.233523    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qbd7m" podUID="23dc984a-2a51-401e-822c-1a67fe34da8e"
Jul 23 19:17:14.233102 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.233056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:14.233102 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.233101    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:14.233328 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:14.233161    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-69v2n" podUID="325ebcee-edbf-49ad-b9a0-ae16bffcb462"
Jul 23 19:17:14.233328 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:14.233214    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5km4c" podUID="514184d7-e116-4ec4-a5b9-b2384ddf0468"
Jul 23 19:17:14.522310 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.522283    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-14.ec2.internal" event="NodeReady"
Jul 23 19:17:14.522865 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.522402    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 19:17:14.603985 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.603947    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xfq4f"]
Jul 23 19:17:14.624775 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.624746    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-946bd"]
Jul 23 19:17:14.624935 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.624875    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.628215 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.628195    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8cbrx\""
Jul 23 19:17:14.628489 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.628472    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 19:17:14.628550 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.628488    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 19:17:14.629416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.629363    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 19:17:14.640866 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.640849    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.643578 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.643563    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gvmps\""
Jul 23 19:17:14.643891 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.643868    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 19:17:14.644001 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.643924    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 19:17:14.649655 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.649635    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xfq4f"]
Jul 23 19:17:14.664257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.664235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-946bd"]
Jul 23 19:17:14.692884 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.692857    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v9fcz"]
Jul 23 19:17:14.719015 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.718992    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v9fcz"]
Jul 23 19:17:14.719142 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.719090    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.722484 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.722462    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 19:17:14.722641 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.722497    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 19:17:14.722740 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.722708    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 19:17:14.722740 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.722708    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 19:17:14.722962 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.722948    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p4fh2\""
Jul 23 19:17:14.728340 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27c16949-2d83-482e-8c4e-0ec67e7a4d38-tmp-dir\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.728433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d3ecef0-850d-41b2-871a-53da311d0f84-crio-socket\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.728433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d3ecef0-850d-41b2-871a-53da311d0f84-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.728433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728398    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0941cb00-878d-4651-8e88-0512be077273-cert\") pod \"ingress-canary-xfq4f\" (UID: \"0941cb00-878d-4651-8e88-0512be077273\") " pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.728540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pkl6\" (UniqueName: \"kubernetes.io/projected/3d3ecef0-850d-41b2-871a-53da311d0f84-kube-api-access-4pkl6\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.728540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27c16949-2d83-482e-8c4e-0ec67e7a4d38-config-volume\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.728540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d3ecef0-850d-41b2-871a-53da311d0f84-data-volume\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.728625 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728544    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27c16949-2d83-482e-8c4e-0ec67e7a4d38-metrics-tls\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.728625 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjjmm\" (UniqueName: \"kubernetes.io/projected/27c16949-2d83-482e-8c4e-0ec67e7a4d38-kube-api-access-zjjmm\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.728625 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728577    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p56gj\" (UniqueName: \"kubernetes.io/projected/0941cb00-878d-4651-8e88-0512be077273-kube-api-access-p56gj\") pod \"ingress-canary-xfq4f\" (UID: \"0941cb00-878d-4651-8e88-0512be077273\") " pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.728625 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.728609    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d3ecef0-850d-41b2-871a-53da311d0f84-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829050 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.828963    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkl6\" (UniqueName: \"kubernetes.io/projected/3d3ecef0-850d-41b2-871a-53da311d0f84-kube-api-access-4pkl6\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829050 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27c16949-2d83-482e-8c4e-0ec67e7a4d38-config-volume\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.829050 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d3ecef0-850d-41b2-871a-53da311d0f84-data-volume\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829050 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27c16949-2d83-482e-8c4e-0ec67e7a4d38-metrics-tls\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.829368 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjjmm\" (UniqueName: \"kubernetes.io/projected/27c16949-2d83-482e-8c4e-0ec67e7a4d38-kube-api-access-zjjmm\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.829368 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p56gj\" (UniqueName: \"kubernetes.io/projected/0941cb00-878d-4651-8e88-0512be077273-kube-api-access-p56gj\") pod \"ingress-canary-xfq4f\" (UID: \"0941cb00-878d-4651-8e88-0512be077273\") " pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.829368 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d3ecef0-850d-41b2-871a-53da311d0f84-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27c16949-2d83-482e-8c4e-0ec67e7a4d38-tmp-dir\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.829520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d3ecef0-850d-41b2-871a-53da311d0f84-data-volume\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d3ecef0-850d-41b2-871a-53da311d0f84-crio-socket\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d3ecef0-850d-41b2-871a-53da311d0f84-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829453    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0941cb00-878d-4651-8e88-0512be077273-cert\") pod \"ingress-canary-xfq4f\" (UID: \"0941cb00-878d-4651-8e88-0512be077273\") " pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.829801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829587    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d3ecef0-850d-41b2-871a-53da311d0f84-crio-socket\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.829801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829646    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27c16949-2d83-482e-8c4e-0ec67e7a4d38-config-volume\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.829930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.829909    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27c16949-2d83-482e-8c4e-0ec67e7a4d38-tmp-dir\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.830123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.830104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d3ecef0-850d-41b2-871a-53da311d0f84-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.833409 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.833390    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d3ecef0-850d-41b2-871a-53da311d0f84-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.833484 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.833429    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27c16949-2d83-482e-8c4e-0ec67e7a4d38-metrics-tls\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.833528 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.833487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0941cb00-878d-4651-8e88-0512be077273-cert\") pod \"ingress-canary-xfq4f\" (UID: \"0941cb00-878d-4651-8e88-0512be077273\") " pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.839327 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.839303    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p56gj\" (UniqueName: \"kubernetes.io/projected/0941cb00-878d-4651-8e88-0512be077273-kube-api-access-p56gj\") pod \"ingress-canary-xfq4f\" (UID: \"0941cb00-878d-4651-8e88-0512be077273\") " pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.839497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.839470    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjjmm\" (UniqueName: \"kubernetes.io/projected/27c16949-2d83-482e-8c4e-0ec67e7a4d38-kube-api-access-zjjmm\") pod \"dns-default-946bd\" (UID: \"27c16949-2d83-482e-8c4e-0ec67e7a4d38\") " pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:14.839562 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.839436    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pkl6\" (UniqueName: \"kubernetes.io/projected/3d3ecef0-850d-41b2-871a-53da311d0f84-kube-api-access-4pkl6\") pod \"insights-runtime-extractor-v9fcz\" (UID: \"3d3ecef0-850d-41b2-871a-53da311d0f84\") " pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:14.936695 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.936645    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xfq4f"
Jul 23 19:17:14.948844 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:14.948816    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:15.027999 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.027967    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v9fcz"
Jul 23 19:17:15.167591 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.166410    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-946bd"]
Jul 23 19:17:15.167591 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.167525    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xfq4f"]
Jul 23 19:17:15.174205 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:15.174175    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c16949_2d83_482e_8c4e_0ec67e7a4d38.slice/crio-3308f5ef13fd587e84cc09852836907050244ee293f56428b986f67aa560d1f6 WatchSource:0}: Error finding container 3308f5ef13fd587e84cc09852836907050244ee293f56428b986f67aa560d1f6: Status 404 returned error can't find the container with id 3308f5ef13fd587e84cc09852836907050244ee293f56428b986f67aa560d1f6
Jul 23 19:17:15.175118 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:15.175095    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0941cb00_878d_4651_8e88_0512be077273.slice/crio-3b35c1e89beeb6d723f3645a4979dbbdd104985c07ab0900a97485f6316ba2b4 WatchSource:0}: Error finding container 3b35c1e89beeb6d723f3645a4979dbbdd104985c07ab0900a97485f6316ba2b4: Status 404 returned error can't find the container with id 3b35c1e89beeb6d723f3645a4979dbbdd104985c07ab0900a97485f6316ba2b4
Jul 23 19:17:15.220507 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.220480    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v9fcz"]
Jul 23 19:17:15.223727 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:15.223703    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3ecef0_850d_41b2_871a_53da311d0f84.slice/crio-89ad889e6a5149e359e15e5156326e97a5186f24bc4194038a5ed8b0268eb76a WatchSource:0}: Error finding container 89ad889e6a5149e359e15e5156326e97a5186f24bc4194038a5ed8b0268eb76a: Status 404 returned error can't find the container with id 89ad889e6a5149e359e15e5156326e97a5186f24bc4194038a5ed8b0268eb76a
Jul 23 19:17:15.233928 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.233903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:15.236692 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.236657    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 19:17:15.236803 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.236785    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6wbwl\""
Jul 23 19:17:15.525493 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.525462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9fcz" event={"ID":"3d3ecef0-850d-41b2-871a-53da311d0f84","Type":"ContainerStarted","Data":"89ad889e6a5149e359e15e5156326e97a5186f24bc4194038a5ed8b0268eb76a"}
Jul 23 19:17:15.526556 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.526534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-946bd" event={"ID":"27c16949-2d83-482e-8c4e-0ec67e7a4d38","Type":"ContainerStarted","Data":"3308f5ef13fd587e84cc09852836907050244ee293f56428b986f67aa560d1f6"}
Jul 23 19:17:15.527624 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:15.527600    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xfq4f" event={"ID":"0941cb00-878d-4651-8e88-0512be077273","Type":"ContainerStarted","Data":"3b35c1e89beeb6d723f3645a4979dbbdd104985c07ab0900a97485f6316ba2b4"}
Jul 23 19:17:16.233396 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:16.233346    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:16.233819 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:16.233802    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:16.237801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:16.237778    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 19:17:16.252048 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:16.252006    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 19:17:16.252345 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:16.252322    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v5npk\""
Jul 23 19:17:16.252597 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:16.252579    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 19:17:18.538341 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:18.538300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-946bd" event={"ID":"27c16949-2d83-482e-8c4e-0ec67e7a4d38","Type":"ContainerStarted","Data":"8b82ef9411c563d3f30a676f98a4b342cdcd53e386ef4d7aa92564dd61b4237a"}
Jul 23 19:17:21.547809 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.547725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-946bd" event={"ID":"27c16949-2d83-482e-8c4e-0ec67e7a4d38","Type":"ContainerStarted","Data":"2bcd467d7d8a84c76e42ce03fc2152bc5592a774aa639dca5a023ac6027c8ff7"}
Jul 23 19:17:21.548244 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.547871    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:21.577821 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.577757    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-946bd" podStartSLOduration=5.077077531 podStartE2EDuration="7.577737031s" totalImagesPullingTime="2.5006595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:21.576702673 +0000 UTC m=+56.865302261" watchObservedRunningTime="2026-07-23 19:17:21.577737031 +0000 UTC m=+56.866336622"
Jul 23 19:17:21.761904 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.761868    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"]
Jul 23 19:17:21.779171 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.779130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.779347 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.779213    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"]
Jul 23 19:17:21.782192 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.782164    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 19:17:21.782192 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.782189    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 19:17:21.782388 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.782226    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 23 19:17:21.782388 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.782189    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mhpg6\""
Jul 23 19:17:21.782532 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.782512    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 19:17:21.782599 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.782559    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 23 19:17:21.883998 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.883907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.883998 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.883975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.884212 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.884004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qhvw\" (UniqueName: \"kubernetes.io/projected/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-kube-api-access-9qhvw\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.884212 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.884080    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.985435 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.985394    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.985598 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.985457    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.985598 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.985504    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.985598 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.985537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qhvw\" (UniqueName: \"kubernetes.io/projected/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-kube-api-access-9qhvw\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.985598 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:21.985561    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 19:17:21.985776 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:21.985626    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-tls podName:c174cc4d-1aa5-4343-91cb-eaec74dbdc71 nodeName:}" failed. No retries permitted until 2026-07-23 19:17:22.485607958 +0000 UTC m=+57.774207544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-dbrv2" (UID: "c174cc4d-1aa5-4343-91cb-eaec74dbdc71") : secret "prometheus-operator-tls" not found
Jul 23 19:17:21.986498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.986469    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.991045 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.991014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:21.997124 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:21.997098    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qhvw\" (UniqueName: \"kubernetes.io/projected/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-kube-api-access-9qhvw\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:22.489935 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.489908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:22.492133 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.492112    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c174cc4d-1aa5-4343-91cb-eaec74dbdc71-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-dbrv2\" (UID: \"c174cc4d-1aa5-4343-91cb-eaec74dbdc71\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:22.551725 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.551691    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9fcz" event={"ID":"3d3ecef0-850d-41b2-871a-53da311d0f84","Type":"ContainerStarted","Data":"8ad5e8ead9f359429d67af5d66be241c8f636ba1feb16298fe0b5835dd07f779"}
Jul 23 19:17:22.553124 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.553097    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xfq4f" event={"ID":"0941cb00-878d-4651-8e88-0512be077273","Type":"ContainerStarted","Data":"76e30675b54135416d2ce5461a1ee2a795487582429fd47b4c40571862342747"}
Jul 23 19:17:22.571399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.571332    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xfq4f" podStartSLOduration=1.673473947 podStartE2EDuration="8.571310865s" totalImagesPullingTime="6.897836918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:22.571244362 +0000 UTC m=+57.859843968" watchObservedRunningTime="2026-07-23 19:17:22.571310865 +0000 UTC m=+57.859910457"
Jul 23 19:17:22.691372 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.691279    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"
Jul 23 19:17:22.837385 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:22.837355    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2"]
Jul 23 19:17:22.840568 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:22.840542    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc174cc4d_1aa5_4343_91cb_eaec74dbdc71.slice/crio-802e788abc3ff976ceb1a1ca3611ee67fa6df8e82a3c7b4e014b0e31409ae70b WatchSource:0}: Error finding container 802e788abc3ff976ceb1a1ca3611ee67fa6df8e82a3c7b4e014b0e31409ae70b: Status 404 returned error can't find the container with id 802e788abc3ff976ceb1a1ca3611ee67fa6df8e82a3c7b4e014b0e31409ae70b
Jul 23 19:17:23.556995 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:23.556930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2" event={"ID":"c174cc4d-1aa5-4343-91cb-eaec74dbdc71","Type":"ContainerStarted","Data":"802e788abc3ff976ceb1a1ca3611ee67fa6df8e82a3c7b4e014b0e31409ae70b"}
Jul 23 19:17:24.476071 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:24.476042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c5mx6"
Jul 23 19:17:26.567422 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:26.567331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9fcz" event={"ID":"3d3ecef0-850d-41b2-871a-53da311d0f84","Type":"ContainerStarted","Data":"4dec42d4231d76c30edba7f1af1cacff3058e60b34595e815c54ee49fcea396d"}
Jul 23 19:17:26.569099 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:26.569059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2" event={"ID":"c174cc4d-1aa5-4343-91cb-eaec74dbdc71","Type":"ContainerStarted","Data":"ca1d659646400a3abf1097caaf4543a0b4b3be390de98011c0f0345df5d1991d"}
Jul 23 19:17:26.569220 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:26.569106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2" event={"ID":"c174cc4d-1aa5-4343-91cb-eaec74dbdc71","Type":"ContainerStarted","Data":"8e6b0b449bf5c330dcabb0ec3fc24d8886d7e9385c5527dcf4fac923f33f75a0"}
Jul 23 19:17:26.588768 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:26.588717    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-dbrv2" podStartSLOduration=2.198253648 podStartE2EDuration="5.588700944s" totalImagesPullingTime="3.390447296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:26.588259396 +0000 UTC m=+61.876858987" watchObservedRunningTime="2026-07-23 19:17:26.588700944 +0000 UTC m=+61.877300513"
Jul 23 19:17:28.173972 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.173938    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"]
Jul 23 19:17:28.176165 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.176141    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.180029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.180004    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 23 19:17:28.180290 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.180270    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 23 19:17:28.180982 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.180960    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bcp5x\""
Jul 23 19:17:28.193953 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.193930    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"]
Jul 23 19:17:28.229753 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.229723    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qdmms"]
Jul 23 19:17:28.232414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.232384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqz8r\" (UniqueName: \"kubernetes.io/projected/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-kube-api-access-nqz8r\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.232565 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.232436    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.232565 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.232459    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.232565 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.232481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.233119 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.233100    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.236404 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.236380    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 23 19:17:28.236733 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.236711    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 23 19:17:28.236842 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.236735    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 23 19:17:28.236842 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.236742    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mt2j5\""
Jul 23 19:17:28.241364 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.241343    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ms8gv"]
Jul 23 19:17:28.243610 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.243593    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.246039 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.246018    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bsg5m\""
Jul 23 19:17:28.246143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.246110    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 19:17:28.246514 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.246492    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 19:17:28.246794 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.246766    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 19:17:28.248267 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.248245    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qdmms"]
Jul 23 19:17:28.333294 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.333294 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.333540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333327    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f63c15f8-6345-4978-80b7-ec354d2d5718-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.333540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333345    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.333540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nbp2\" (UniqueName: \"kubernetes.io/projected/27c57fdc-3ab3-460a-ba96-1215b8c83938-kube-api-access-9nbp2\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.333540 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:28.333422    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 23 19:17:28.333540 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333467    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c57fdc-3ab3-460a-ba96-1215b8c83938-metrics-client-ca\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.333540 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:28.333497    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls podName:21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c nodeName:}" failed. No retries permitted until 2026-07-23 19:17:28.833476311 +0000 UTC m=+64.122075884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-bgkmn" (UID: "21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c") : secret "openshift-state-metrics-tls" not found
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqz8r\" (UniqueName: \"kubernetes.io/projected/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-kube-api-access-nqz8r\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-textfile\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-wtmp\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333825    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-root\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.333853 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-sys\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.334218 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-accelerators-collector-config\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.334218 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333913    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f63c15f8-6345-4978-80b7-ec354d2d5718-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.334218 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333959    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-tls\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.334218 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.333993    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbgm2\" (UniqueName: \"kubernetes.io/projected/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-api-access-zbgm2\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.334218 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.334043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.334406 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.334369    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.336427 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.336382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.343392 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.343363    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqz8r\" (UniqueName: \"kubernetes.io/projected/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-kube-api-access-nqz8r\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.435231 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.435231 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-textfile\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435231 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.435509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-wtmp\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-root\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-sys\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-accelerators-collector-config\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435451    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-wtmp\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-root\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435551    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27c57fdc-3ab3-460a-ba96-1215b8c83938-sys\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435571    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f63c15f8-6345-4978-80b7-ec354d2d5718-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435590    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-textfile\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-tls\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbgm2\" (UniqueName: \"kubernetes.io/projected/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-api-access-zbgm2\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f63c15f8-6345-4978-80b7-ec354d2d5718-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.435827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nbp2\" (UniqueName: \"kubernetes.io/projected/27c57fdc-3ab3-460a-ba96-1215b8c83938-kube-api-access-9nbp2\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.436257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c57fdc-3ab3-460a-ba96-1215b8c83938-metrics-client-ca\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.436257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.435983    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-accelerators-collector-config\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.436257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.436148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.436257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.436248    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f63c15f8-6345-4978-80b7-ec354d2d5718-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.436427 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.436391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c57fdc-3ab3-460a-ba96-1215b8c83938-metrics-client-ca\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.436963 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.436871    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f63c15f8-6345-4978-80b7-ec354d2d5718-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.438588 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.438564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-tls\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.438718 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.438569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.438788 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.438752    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c57fdc-3ab3-460a-ba96-1215b8c83938-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.438934 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.438910    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.447088 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.447043    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbgm2\" (UniqueName: \"kubernetes.io/projected/f63c15f8-6345-4978-80b7-ec354d2d5718-kube-api-access-zbgm2\") pod \"kube-state-metrics-56c5594674-qdmms\" (UID: \"f63c15f8-6345-4978-80b7-ec354d2d5718\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.447252 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.447224    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nbp2\" (UniqueName: \"kubernetes.io/projected/27c57fdc-3ab3-460a-ba96-1215b8c83938-kube-api-access-9nbp2\") pod \"node-exporter-ms8gv\" (UID: \"27c57fdc-3ab3-460a-ba96-1215b8c83938\") " pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.544110 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.544075    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms"
Jul 23 19:17:28.553785 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.553763    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ms8gv"
Jul 23 19:17:28.555800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.555782    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-946bd"
Jul 23 19:17:28.844686 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:28.844588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:28.844837 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:28.844747    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 23 19:17:28.844837 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:28.844823    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls podName:21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c nodeName:}" failed. No retries permitted until 2026-07-23 19:17:29.844803147 +0000 UTC m=+65.133402721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-bgkmn" (UID: "21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c") : secret "openshift-state-metrics-tls" not found
Jul 23 19:17:29.346943 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.346908    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 19:17:29.362800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.362771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.366483 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.366057    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 19:17:29.366483 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.366207    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 19:17:29.366483 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.366357    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 19:17:29.366483 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.366465    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 19:17:29.367574 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.367551    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 19:17:29.367808 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.367775    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 19:17:29.367957 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.367836    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 19:17:29.368038 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.368011    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 19:17:29.368100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.368068    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sfsv9\""
Jul 23 19:17:29.368260 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.368240    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 19:17:29.370129 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.370106    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 19:17:29.450443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450411    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvjlm\" (UniqueName: \"kubernetes.io/projected/df8184d7-bfab-46ef-bc1e-8a9dda541093-kube-api-access-zvjlm\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450632 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450632 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450632 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450599    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-web-config\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450711    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-config-volume\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8184d7-bfab-46ef-bc1e-8a9dda541093-config-out\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450767    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450792    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.450824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8184d7-bfab-46ef-bc1e-8a9dda541093-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.451074 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.451074 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450881    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.451074 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.450905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551324 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551292    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551494 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551353    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-web-config\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551494 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-config-volume\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551494 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8184d7-bfab-46ef-bc1e-8a9dda541093-config-out\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551494 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551492    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8184d7-bfab-46ef-bc1e-8a9dda541093-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551610    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvjlm\" (UniqueName: \"kubernetes.io/projected/df8184d7-bfab-46ef-bc1e-8a9dda541093-kube-api-access-zvjlm\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551739    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551739    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.551882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.551788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.552862 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.552625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.553693 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:29.553189    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-trusted-ca-bundle podName:df8184d7-bfab-46ef-bc1e-8a9dda541093 nodeName:}" failed. No retries permitted until 2026-07-23 19:17:30.053167855 +0000 UTC m=+65.341767427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "df8184d7-bfab-46ef-bc1e-8a9dda541093") : configmap references non-existent config key: ca-bundle.crt
Jul 23 19:17:29.555162 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.555145    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-web-config\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.555874 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.555849    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df8184d7-bfab-46ef-bc1e-8a9dda541093-config-out\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.556005 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.555985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df8184d7-bfab-46ef-bc1e-8a9dda541093-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.556229 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.556205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.556375 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.556357    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.556457 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.556441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.556894 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.556868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.557147 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.557106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-config-volume\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.558440 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.558420    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df8184d7-bfab-46ef-bc1e-8a9dda541093-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.562787 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.562749    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvjlm\" (UniqueName: \"kubernetes.io/projected/df8184d7-bfab-46ef-bc1e-8a9dda541093-kube-api-access-zvjlm\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:29.842783 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:29.842747    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c57fdc_3ab3_460a_ba96_1215b8c83938.slice/crio-227fafc84562f455ce6e2558c921761408d55a71df82ec5fd32500980a3c1752 WatchSource:0}: Error finding container 227fafc84562f455ce6e2558c921761408d55a71df82ec5fd32500980a3c1752: Status 404 returned error can't find the container with id 227fafc84562f455ce6e2558c921761408d55a71df82ec5fd32500980a3c1752
Jul 23 19:17:29.854332 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.854142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:29.856802 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.856773    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-bgkmn\" (UID: \"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:29.955437 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.955407    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:29.955550 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.955456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:29.957556 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.957536    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 19:17:29.957871 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.957851    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 19:17:29.968206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.968172    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/514184d7-e116-4ec4-a5b9-b2384ddf0468-original-pull-secret\") pod \"global-pull-secret-syncer-5km4c\" (UID: \"514184d7-e116-4ec4-a5b9-b2384ddf0468\") " pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:29.968319 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.968303    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23dc984a-2a51-401e-822c-1a67fe34da8e-metrics-certs\") pod \"network-metrics-daemon-qbd7m\" (UID: \"23dc984a-2a51-401e-822c-1a67fe34da8e\") " pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:29.973890 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.973868    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qdmms"]
Jul 23 19:17:29.976473 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:29.976453    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63c15f8_6345_4978_80b7_ec354d2d5718.slice/crio-4fdb1d001e5d8e1c4569150d75ba0ace00bace88a70f2038aedc9cfbbf93c319 WatchSource:0}: Error finding container 4fdb1d001e5d8e1c4569150d75ba0ace00bace88a70f2038aedc9cfbbf93c319: Status 404 returned error can't find the container with id 4fdb1d001e5d8e1c4569150d75ba0ace00bace88a70f2038aedc9cfbbf93c319
Jul 23 19:17:29.985587 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:29.985566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"
Jul 23 19:17:30.056536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.056503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:30.056536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.056538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:30.057472 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.057452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df8184d7-bfab-46ef-bc1e-8a9dda541093-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df8184d7-bfab-46ef-bc1e-8a9dda541093\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:30.059175 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.059151    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 19:17:30.070210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.070192    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5km4c"
Jul 23 19:17:30.070210 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.070211    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 19:17:30.080553 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.080533    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9nr\" (UniqueName: \"kubernetes.io/projected/325ebcee-edbf-49ad-b9a0-ae16bffcb462-kube-api-access-xw9nr\") pod \"network-check-target-69v2n\" (UID: \"325ebcee-edbf-49ad-b9a0-ae16bffcb462\") " pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:30.113489 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.113401    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn"]
Jul 23 19:17:30.117537 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:30.117503    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b6ce29_a1b2_4a7e_9059_cb3ed4097c4c.slice/crio-d5266f758e9a39227ac14a634c3c2e5c07e65dcb2415c4aea93bcc8b48735e4b WatchSource:0}: Error finding container d5266f758e9a39227ac14a634c3c2e5c07e65dcb2415c4aea93bcc8b48735e4b: Status 404 returned error can't find the container with id d5266f758e9a39227ac14a634c3c2e5c07e65dcb2415c4aea93bcc8b48735e4b
Jul 23 19:17:30.207604 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.207579    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5km4c"]
Jul 23 19:17:30.213660 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:30.213634    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514184d7_e116_4ec4_a5b9_b2384ddf0468.slice/crio-3b2fbdf3d9730bd1008e58ff45e01ec0f191a18a0619a1b3051f8abcf0be4852 WatchSource:0}: Error finding container 3b2fbdf3d9730bd1008e58ff45e01ec0f191a18a0619a1b3051f8abcf0be4852: Status 404 returned error can't find the container with id 3b2fbdf3d9730bd1008e58ff45e01ec0f191a18a0619a1b3051f8abcf0be4852
Jul 23 19:17:30.246190 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.246170    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6wbwl\""
Jul 23 19:17:30.254308 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.254286    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qbd7m"
Jul 23 19:17:30.274718 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.274693    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 19:17:30.383416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.383356    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v5npk\""
Jul 23 19:17:30.390575 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.390421    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:30.405340 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.404053    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qbd7m"]
Jul 23 19:17:30.438775 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.438661    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 19:17:30.544421 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.544382    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-69v2n"]
Jul 23 19:17:30.548239 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:30.548210    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325ebcee_edbf_49ad_b9a0_ae16bffcb462.slice/crio-0eef15e4f91e645cf4f406d47bfada57fa230aaf6f1a1a5c944ca9efbb8923bb WatchSource:0}: Error finding container 0eef15e4f91e645cf4f406d47bfada57fa230aaf6f1a1a5c944ca9efbb8923bb: Status 404 returned error can't find the container with id 0eef15e4f91e645cf4f406d47bfada57fa230aaf6f1a1a5c944ca9efbb8923bb
Jul 23 19:17:30.587655 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.587611    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9fcz" event={"ID":"3d3ecef0-850d-41b2-871a-53da311d0f84","Type":"ContainerStarted","Data":"4266745f5142893b8b538f1939d06ad7ec22c760fb8b7bb4d6125ef794863a46"}
Jul 23 19:17:30.588970 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.588935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"9dcb4294c7ff4ef64dbe0abd93dd1056419c3b2291e5fab37b181072331f59ad"}
Jul 23 19:17:30.590192 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.590170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qbd7m" event={"ID":"23dc984a-2a51-401e-822c-1a67fe34da8e","Type":"ContainerStarted","Data":"22aabb741d85eba594586b7e64ffba73537357f0c90a893b797bd4ab2e9c806a"}
Jul 23 19:17:30.591436 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.591411    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-69v2n" event={"ID":"325ebcee-edbf-49ad-b9a0-ae16bffcb462","Type":"ContainerStarted","Data":"0eef15e4f91e645cf4f406d47bfada57fa230aaf6f1a1a5c944ca9efbb8923bb"}
Jul 23 19:17:30.593600 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.593479    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn" event={"ID":"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c","Type":"ContainerStarted","Data":"854f2a4c3d5ec035d0c4df1f822dbf4c6f137bd3f692a71722e649448108d2fb"}
Jul 23 19:17:30.593600 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.593540    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn" event={"ID":"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c","Type":"ContainerStarted","Data":"5957d7afa4ab8d4a250b33d044ae25e354bb99ec5eed41e5be31c2817584375d"}
Jul 23 19:17:30.593600 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.593570    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn" event={"ID":"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c","Type":"ContainerStarted","Data":"d5266f758e9a39227ac14a634c3c2e5c07e65dcb2415c4aea93bcc8b48735e4b"}
Jul 23 19:17:30.594730 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.594706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ms8gv" event={"ID":"27c57fdc-3ab3-460a-ba96-1215b8c83938","Type":"ContainerStarted","Data":"227fafc84562f455ce6e2558c921761408d55a71df82ec5fd32500980a3c1752"}
Jul 23 19:17:30.595841 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.595822    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5km4c" event={"ID":"514184d7-e116-4ec4-a5b9-b2384ddf0468","Type":"ContainerStarted","Data":"3b2fbdf3d9730bd1008e58ff45e01ec0f191a18a0619a1b3051f8abcf0be4852"}
Jul 23 19:17:30.596941 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.596924    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms" event={"ID":"f63c15f8-6345-4978-80b7-ec354d2d5718","Type":"ContainerStarted","Data":"4fdb1d001e5d8e1c4569150d75ba0ace00bace88a70f2038aedc9cfbbf93c319"}
Jul 23 19:17:30.608521 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:30.608472    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v9fcz" podStartSLOduration=2.134240025 podStartE2EDuration="16.608456031s" totalImagesPullingTime="14.474216006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:17:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:30.607511391 +0000 UTC m=+65.896110984" watchObservedRunningTime="2026-07-23 19:17:30.608456031 +0000 UTC m=+65.897055626"
Jul 23 19:17:31.288569 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.288535    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"]
Jul 23 19:17:31.291529 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.291508    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.295352 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.295330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q4qfk\""
Jul 23 19:17:31.295784 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.295714    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 23 19:17:31.295960 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.295913    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 23 19:17:31.296102 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.296084    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 23 19:17:31.296293 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.296274    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 23 19:17:31.298111 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.298087    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 23 19:17:31.302635 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.302610    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dq80qm8fb6gp2\""
Jul 23 19:17:31.346867 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.346840    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"]
Jul 23 19:17:31.366433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366403    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-grpc-tls\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.366621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366464    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.366621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4t9\" (UniqueName: \"kubernetes.io/projected/063fda30-5178-419b-9ef9-9e8d3d56fbb8-kube-api-access-bj4t9\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.366621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366534    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/063fda30-5178-419b-9ef9-9e8d3d56fbb8-metrics-client-ca\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.366621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366562    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.366621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366614    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.367115 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366653    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.367115 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.366692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-tls\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467461    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-tls\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467511    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-grpc-tls\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4t9\" (UniqueName: \"kubernetes.io/projected/063fda30-5178-419b-9ef9-9e8d3d56fbb8-kube-api-access-bj4t9\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/063fda30-5178-419b-9ef9-9e8d3d56fbb8-metrics-client-ca\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467706    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.468143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.467760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.470266 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.469714    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/063fda30-5178-419b-9ef9-9e8d3d56fbb8-metrics-client-ca\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.472174 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.472146    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.472920 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.472826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.473509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.473465    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-tls\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.474906 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.474851    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.474906 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.474875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-grpc-tls\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.475061 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.474921    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/063fda30-5178-419b-9ef9-9e8d3d56fbb8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.484154 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.484104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4t9\" (UniqueName: \"kubernetes.io/projected/063fda30-5178-419b-9ef9-9e8d3d56fbb8-kube-api-access-bj4t9\") pod \"thanos-querier-cfcccfcf9-dg4l2\" (UID: \"063fda30-5178-419b-9ef9-9e8d3d56fbb8\") " pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:31.604124 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.603358    2566 generic.go:356] "Generic (PLEG): container finished" podID="27c57fdc-3ab3-460a-ba96-1215b8c83938" containerID="c2452a3cbf72247562d83e3ecdec04d79abcc231c8eb1d6a6c5a089443b58a68" exitCode=0
Jul 23 19:17:31.604124 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.603989    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ms8gv" event={"ID":"27c57fdc-3ab3-460a-ba96-1215b8c83938","Type":"ContainerDied","Data":"c2452a3cbf72247562d83e3ecdec04d79abcc231c8eb1d6a6c5a089443b58a68"}
Jul 23 19:17:31.606142 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:31.606120    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:32.724772 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.724524    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f495dc744-qx6s5"]
Jul 23 19:17:32.728311 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.728287    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.733941 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.733919    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 19:17:32.734061 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.733950    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 19:17:32.734130 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.734087    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 19:17:32.734237 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.734215    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x4dhj\""
Jul 23 19:17:32.734345 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.734331    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d38k6k9q61q7r\""
Jul 23 19:17:32.734516 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.734497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 19:17:32.759689 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.759643    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f495dc744-qx6s5"]
Jul 23 19:17:32.781184 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781150    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zhrd\" (UniqueName: \"kubernetes.io/projected/fc141d97-e7fd-49f8-b878-5a7eea218ce2-kube-api-access-2zhrd\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.781314 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc141d97-e7fd-49f8-b878-5a7eea218ce2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.781314 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-client-ca-bundle\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.781479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781318    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fc141d97-e7fd-49f8-b878-5a7eea218ce2-audit-log\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.781479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-secret-metrics-server-tls\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.781479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781366    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fc141d97-e7fd-49f8-b878-5a7eea218ce2-metrics-server-audit-profiles\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.781479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.781411    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-secret-metrics-server-client-certs\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.882615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.882231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zhrd\" (UniqueName: \"kubernetes.io/projected/fc141d97-e7fd-49f8-b878-5a7eea218ce2-kube-api-access-2zhrd\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.882615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.882317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc141d97-e7fd-49f8-b878-5a7eea218ce2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.882615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.882376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-client-ca-bundle\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.882615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.882405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fc141d97-e7fd-49f8-b878-5a7eea218ce2-audit-log\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.883848 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.883213    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc141d97-e7fd-49f8-b878-5a7eea218ce2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.884399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.884003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-secret-metrics-server-tls\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.884399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.884035    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fc141d97-e7fd-49f8-b878-5a7eea218ce2-metrics-server-audit-profiles\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.884399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.884067    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-secret-metrics-server-client-certs\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.884399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.884272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fc141d97-e7fd-49f8-b878-5a7eea218ce2-audit-log\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.885468 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.885182    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fc141d97-e7fd-49f8-b878-5a7eea218ce2-metrics-server-audit-profiles\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.887166 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.887143    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-secret-metrics-server-tls\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.889009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.888969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-client-ca-bundle\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.893029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.892848    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fc141d97-e7fd-49f8-b878-5a7eea218ce2-secret-metrics-server-client-certs\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.893405 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.893384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zhrd\" (UniqueName: \"kubernetes.io/projected/fc141d97-e7fd-49f8-b878-5a7eea218ce2-kube-api-access-2zhrd\") pod \"metrics-server-6f495dc744-qx6s5\" (UID: \"fc141d97-e7fd-49f8-b878-5a7eea218ce2\") " pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:32.922344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.922270    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f9d57f47f-8gb2x"]
Jul 23 19:17:32.924725 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.924705    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.927992    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.928235    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.928378    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.928764    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.928914    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.929083    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 19:17:32.929631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.929388    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xmvpc\""
Jul 23 19:17:32.930179 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.930035    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 19:17:32.937717 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.936454    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"]
Jul 23 19:17:32.937717 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.937243    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 19:17:32.939185 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.938792    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:32.940072 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.940053    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9d57f47f-8gb2x"]
Jul 23 19:17:32.941375 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.941353    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 23 19:17:32.941471 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.941357    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jxrzj\""
Jul 23 19:17:32.957428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.957406    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"]
Jul 23 19:17:32.984788 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984743    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-config\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.984932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984795    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b7569b2-3f22-4015-b8ad-9111642391b4-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:32.984932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tqmj\" (UniqueName: \"kubernetes.io/projected/9b7569b2-3f22-4015-b8ad-9111642391b4-kube-api-access-5tqmj\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:32.984932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984860    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-trusted-ca-bundle\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.984932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984906    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-serving-cert\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.984932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-oauth-serving-cert\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.985123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984948    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-service-ca\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.985123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984962    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-oauth-config\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:32.985123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:32.984992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzv2\" (UniqueName: \"kubernetes.io/projected/c32f919d-9cca-434c-9cd0-4a1de158d12d-kube-api-access-zvzv2\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.041485 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.041404    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:33.085874 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.085842    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-serving-cert\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.085874 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.085887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-oauth-serving-cert\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.086077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.085924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-service-ca\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.086077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.085949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-oauth-config\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.086077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.085991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzv2\" (UniqueName: \"kubernetes.io/projected/c32f919d-9cca-434c-9cd0-4a1de158d12d-kube-api-access-zvzv2\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.086077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.086029    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-config\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.086077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.086056    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b7569b2-3f22-4015-b8ad-9111642391b4-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:33.086326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.086082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tqmj\" (UniqueName: \"kubernetes.io/projected/9b7569b2-3f22-4015-b8ad-9111642391b4-kube-api-access-5tqmj\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:33.086326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.086124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-trusted-ca-bundle\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.087709 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:33.086533    2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 23 19:17:33.087709 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:17:33.086617    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7569b2-3f22-4015-b8ad-9111642391b4-monitoring-plugin-cert podName:9b7569b2-3f22-4015-b8ad-9111642391b4 nodeName:}" failed. No retries permitted until 2026-07-23 19:17:33.586595899 +0000 UTC m=+68.875195472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9b7569b2-3f22-4015-b8ad-9111642391b4-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-jwckb" (UID: "9b7569b2-3f22-4015-b8ad-9111642391b4") : secret "monitoring-plugin-cert" not found
Jul 23 19:17:33.087709 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.087103    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-trusted-ca-bundle\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.087709 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.087302    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-config\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.087709 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.087394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-oauth-serving-cert\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.087995 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.087809    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-service-ca\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.089891 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.089868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-serving-cert\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.092426 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.092401    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-oauth-config\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.097002 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.096956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tqmj\" (UniqueName: \"kubernetes.io/projected/9b7569b2-3f22-4015-b8ad-9111642391b4-kube-api-access-5tqmj\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:33.099713 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.099692    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzv2\" (UniqueName: \"kubernetes.io/projected/c32f919d-9cca-434c-9cd0-4a1de158d12d-kube-api-access-zvzv2\") pod \"console-6f9d57f47f-8gb2x\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") " pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.236535 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.236508    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:33.459195 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.459155    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-757d74c77c-rk76j"]
Jul 23 19:17:33.461890 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.461869    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.465314 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.465291    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-824hs\""
Jul 23 19:17:33.465456 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.465301    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 23 19:17:33.465552 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.465537    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 23 19:17:33.465618 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.465562    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 23 19:17:33.465845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.465829    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 23 19:17:33.465946 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.465871    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 23 19:17:33.471930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.471909    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 23 19:17:33.477280 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.477243    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-757d74c77c-rk76j"]
Jul 23 19:17:33.589722 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-telemeter-trusted-ca-bundle\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.589909 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589733    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-serving-certs-ca-bundle\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.589909 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589764    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.589909 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589892    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-metrics-client-ca\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.590069 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-secret-telemeter-client\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.590069 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-telemeter-client-tls\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.590069 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.589990    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-federate-client-tls\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.590170 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.590070    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b7569b2-3f22-4015-b8ad-9111642391b4-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:33.590170 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.590114    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crj2h\" (UniqueName: \"kubernetes.io/projected/f2ff0639-7b16-49d9-a210-2edce75e9233-kube-api-access-crj2h\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.592856 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.592835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b7569b2-3f22-4015-b8ad-9111642391b4-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwckb\" (UID: \"9b7569b2-3f22-4015-b8ad-9111642391b4\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:33.690983 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.690949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crj2h\" (UniqueName: \"kubernetes.io/projected/f2ff0639-7b16-49d9-a210-2edce75e9233-kube-api-access-crj2h\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691180 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.690997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-telemeter-trusted-ca-bundle\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691180 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.691040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-serving-certs-ca-bundle\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691180 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.691062    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691180 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.691135    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-metrics-client-ca\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691180 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.691166    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-secret-telemeter-client\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.691190    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-telemeter-client-tls\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.691416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.691207    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-federate-client-tls\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.692178 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.692153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-metrics-client-ca\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.692302 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.692257    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-telemeter-trusted-ca-bundle\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.692409 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.692300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ff0639-7b16-49d9-a210-2edce75e9233-serving-certs-ca-bundle\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.693993 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.693970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-federate-client-tls\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.694335 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.694316    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.694781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.694752    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-telemeter-client-tls\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.694874 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.694823    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f2ff0639-7b16-49d9-a210-2edce75e9233-secret-telemeter-client\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.703029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.703006    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crj2h\" (UniqueName: \"kubernetes.io/projected/f2ff0639-7b16-49d9-a210-2edce75e9233-kube-api-access-crj2h\") pod \"telemeter-client-757d74c77c-rk76j\" (UID: \"f2ff0639-7b16-49d9-a210-2edce75e9233\") " pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.773248 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.773153    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j"
Jul 23 19:17:33.850145 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:33.850106    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:34.093009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.092973    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"]
Jul 23 19:17:34.447762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.447722    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 19:17:34.458747 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.458716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.462764 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.462739    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 19:17:34.463010 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.462994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 19:17:34.463208 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.463189    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h5gqt\""
Jul 23 19:17:34.463390 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.463372    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 19:17:34.464820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.463565    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 19:17:34.464820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.463740    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 19:17:34.464820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.463886    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8tpk6mr68hfki\""
Jul 23 19:17:34.464820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.464058    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 19:17:34.464820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.464515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 19:17:34.465183 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.465161    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 19:17:34.465388 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.465373    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 19:17:34.467123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.465578    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 19:17:34.467123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.465797    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 19:17:34.467123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.465977    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 19:17:34.470327 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.469089    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 19:17:34.471096 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.470859    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 19:17:34.475929 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.475907    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 19:17:34.602042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64a38aba-3f39-4bf7-85f0-da9ab233960e-config-out\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602116    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602136    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-web-config\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602430 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602229    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602430 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602277    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602430 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602308    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602430 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602430 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-config\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602432    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602501    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602546    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602598    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64a38aba-3f39-4bf7-85f0-da9ab233960e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqwv\" (UniqueName: \"kubernetes.io/projected/64a38aba-3f39-4bf7-85f0-da9ab233960e-kube-api-access-dwqwv\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.602800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.602687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.704597 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.704492    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.704597 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.704549    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705087    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705135    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64a38aba-3f39-4bf7-85f0-da9ab233960e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705199    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqwv\" (UniqueName: \"kubernetes.io/projected/64a38aba-3f39-4bf7-85f0-da9ab233960e-kube-api-access-dwqwv\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705230    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705261    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705295    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64a38aba-3f39-4bf7-85f0-da9ab233960e-config-out\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705319    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-web-config\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705400    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705435    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.705703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.705473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.707119 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.706591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.707119 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.706712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.707119 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.706749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-config\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.707119 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.706782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.707607 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.707583    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.708409 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.708384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.709479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.708653    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.709479 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.709189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.710699 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.709983    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64a38aba-3f39-4bf7-85f0-da9ab233960e-config-out\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.710699 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.710388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.710699 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.710545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.711520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.711481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.712047 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.711987    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64a38aba-3f39-4bf7-85f0-da9ab233960e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.712806 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.712138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.712939 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.712772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/64a38aba-3f39-4bf7-85f0-da9ab233960e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.713822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.713094    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.713822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.713164    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.713822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.713692    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.713822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.713699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-config\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.713822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.713760    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.714140 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.714121    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.715239 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.714272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.715239 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.714523    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64a38aba-3f39-4bf7-85f0-da9ab233960e-web-config\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.722214 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.722197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqwv\" (UniqueName: \"kubernetes.io/projected/64a38aba-3f39-4bf7-85f0-da9ab233960e-kube-api-access-dwqwv\") pod \"prometheus-k8s-0\" (UID: \"64a38aba-3f39-4bf7-85f0-da9ab233960e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:34.775444 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:34.775412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:35.577225 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:35.577181    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063fda30_5178_419b_9ef9_9e8d3d56fbb8.slice/crio-bc897399ba0d4f771d331cf6d7eedbc98a8bebdc386014b83c54a2164095e25b WatchSource:0}: Error finding container bc897399ba0d4f771d331cf6d7eedbc98a8bebdc386014b83c54a2164095e25b: Status 404 returned error can't find the container with id bc897399ba0d4f771d331cf6d7eedbc98a8bebdc386014b83c54a2164095e25b
Jul 23 19:17:35.621693 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:35.621645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"bc897399ba0d4f771d331cf6d7eedbc98a8bebdc386014b83c54a2164095e25b"}
Jul 23 19:17:35.897279 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:35.896621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-757d74c77c-rk76j"]
Jul 23 19:17:35.950814 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:35.950515    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f495dc744-qx6s5"]
Jul 23 19:17:35.965641 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:35.965605    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc141d97_e7fd_49f8_b878_5a7eea218ce2.slice/crio-632bf6b1fb548514c966ccd287a762cbaa36852b1db094e83c717a0e8c17c903 WatchSource:0}: Error finding container 632bf6b1fb548514c966ccd287a762cbaa36852b1db094e83c717a0e8c17c903: Status 404 returned error can't find the container with id 632bf6b1fb548514c966ccd287a762cbaa36852b1db094e83c717a0e8c17c903
Jul 23 19:17:36.123499 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.123434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"]
Jul 23 19:17:36.125885 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.125830    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9d57f47f-8gb2x"]
Jul 23 19:17:36.126961 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:36.126925    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b7569b2_3f22_4015_b8ad_9111642391b4.slice/crio-bb720e42f0735405703a80e5549b6ebe51f456651157f850aa6f131c14269626 WatchSource:0}: Error finding container bb720e42f0735405703a80e5549b6ebe51f456651157f850aa6f131c14269626: Status 404 returned error can't find the container with id bb720e42f0735405703a80e5549b6ebe51f456651157f850aa6f131c14269626
Jul 23 19:17:36.133223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.133201    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 19:17:36.134765 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:36.134687    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32f919d_9cca_434c_9cd0_4a1de158d12d.slice/crio-d7874fd9b453d609b6cffbf523e28000045ad2b9bbf19b6322eeab556e1e36fb WatchSource:0}: Error finding container d7874fd9b453d609b6cffbf523e28000045ad2b9bbf19b6322eeab556e1e36fb: Status 404 returned error can't find the container with id d7874fd9b453d609b6cffbf523e28000045ad2b9bbf19b6322eeab556e1e36fb
Jul 23 19:17:36.137888 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:17:36.137775    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a38aba_3f39_4bf7_85f0_da9ab233960e.slice/crio-79c4418408e218b2274a1f43a56d91f5b3227ed862039c97d9499b0195823d93 WatchSource:0}: Error finding container 79c4418408e218b2274a1f43a56d91f5b3227ed862039c97d9499b0195823d93: Status 404 returned error can't find the container with id 79c4418408e218b2274a1f43a56d91f5b3227ed862039c97d9499b0195823d93
Jul 23 19:17:36.632132 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.630962    2566 generic.go:356] "Generic (PLEG): container finished" podID="64a38aba-3f39-4bf7-85f0-da9ab233960e" containerID="1fc0e6e64c04c0bb0ce533e8c48a2a834536b0e9ecf4794690951eab62a63c6a" exitCode=0
Jul 23 19:17:36.632132 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.631047    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerDied","Data":"1fc0e6e64c04c0bb0ce533e8c48a2a834536b0e9ecf4794690951eab62a63c6a"}
Jul 23 19:17:36.632132 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.631119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"79c4418408e218b2274a1f43a56d91f5b3227ed862039c97d9499b0195823d93"}
Jul 23 19:17:36.635610 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.635529    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j" event={"ID":"f2ff0639-7b16-49d9-a210-2edce75e9233","Type":"ContainerStarted","Data":"dd023dc506a23e463016a9688086e8670b1f0c5841091f3ae09ce218d037d052"}
Jul 23 19:17:36.637690 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.637646    2566 generic.go:356] "Generic (PLEG): container finished" podID="df8184d7-bfab-46ef-bc1e-8a9dda541093" containerID="00b6912bc49860697c0390f52a307cd95ee92ee2c22700ac119e9db8b9a61378" exitCode=0
Jul 23 19:17:36.637811 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.637761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerDied","Data":"00b6912bc49860697c0390f52a307cd95ee92ee2c22700ac119e9db8b9a61378"}
Jul 23 19:17:36.640206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.640172    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qbd7m" event={"ID":"23dc984a-2a51-401e-822c-1a67fe34da8e","Type":"ContainerStarted","Data":"5cfaefc167b25d71a67952402d8bad080de8c31109f771e735a1e5dd1e96c5d0"}
Jul 23 19:17:36.640300 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.640234    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qbd7m" event={"ID":"23dc984a-2a51-401e-822c-1a67fe34da8e","Type":"ContainerStarted","Data":"1d2c1dbbeb849586b00d3ae3690fc8cec053202b13f3d14896ab94d39a820067"}
Jul 23 19:17:36.644132 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.644104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-69v2n" event={"ID":"325ebcee-edbf-49ad-b9a0-ae16bffcb462","Type":"ContainerStarted","Data":"772caaa9003406e9467ffce26dff374c26d5d1aca0b0889a864a716fdcfe5134"}
Jul 23 19:17:36.644563 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.644541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:17:36.648285 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.647738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn" event={"ID":"21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c","Type":"ContainerStarted","Data":"3611797d9ef1931ec9151ba67e1e96372ce6944dff29fb93923935d8e065f3be"}
Jul 23 19:17:36.653217 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.653142    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ms8gv" event={"ID":"27c57fdc-3ab3-460a-ba96-1215b8c83938","Type":"ContainerStarted","Data":"24817a35e411824ff3f175786da826d948dd4b67877cc35ac3b133f30923c4cc"}
Jul 23 19:17:36.653217 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.653187    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ms8gv" event={"ID":"27c57fdc-3ab3-460a-ba96-1215b8c83938","Type":"ContainerStarted","Data":"ec3ff617b43b27efdda8267960569632b530d70caa8ce344153066be26735de6"}
Jul 23 19:17:36.654980 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.654946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb" event={"ID":"9b7569b2-3f22-4015-b8ad-9111642391b4","Type":"ContainerStarted","Data":"bb720e42f0735405703a80e5549b6ebe51f456651157f850aa6f131c14269626"}
Jul 23 19:17:36.656609 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.656571    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9d57f47f-8gb2x" event={"ID":"c32f919d-9cca-434c-9cd0-4a1de158d12d","Type":"ContainerStarted","Data":"d7874fd9b453d609b6cffbf523e28000045ad2b9bbf19b6322eeab556e1e36fb"}
Jul 23 19:17:36.659705 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.659559    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5km4c" event={"ID":"514184d7-e116-4ec4-a5b9-b2384ddf0468","Type":"ContainerStarted","Data":"ed9f206b8bc10663b3870b7e0b198e7f7b5f1aee4ef94fce712ab0b9c35272b3"}
Jul 23 19:17:36.662328 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.662307    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms" event={"ID":"f63c15f8-6345-4978-80b7-ec354d2d5718","Type":"ContainerStarted","Data":"5aed535ba6d5d2c3d970b2367638a7e01504886e884b15fad5a3fa85efa00dd6"}
Jul 23 19:17:36.662424 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.662358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms" event={"ID":"f63c15f8-6345-4978-80b7-ec354d2d5718","Type":"ContainerStarted","Data":"f71951603e2cd83e8394f2d0f5b2630f85b1b8df39567b73a03d88a540abeaa7"}
Jul 23 19:17:36.662424 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.662389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms" event={"ID":"f63c15f8-6345-4978-80b7-ec354d2d5718","Type":"ContainerStarted","Data":"91e836a595492a85573423f8dd287290d762fd24281a0d57bba7676f5856ac95"}
Jul 23 19:17:36.664834 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.664534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5" event={"ID":"fc141d97-e7fd-49f8-b878-5a7eea218ce2","Type":"ContainerStarted","Data":"632bf6b1fb548514c966ccd287a762cbaa36852b1db094e83c717a0e8c17c903"}
Jul 23 19:17:36.688151 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.688100    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-qdmms" podStartSLOduration=3.032812424 podStartE2EDuration="8.688079761s" totalImagesPullingTime="5.655267337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:36.687324781 +0000 UTC m=+71.975924376" watchObservedRunningTime="2026-07-23 19:17:36.688079761 +0000 UTC m=+71.976679349"
Jul 23 19:17:36.715152 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.714935    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ms8gv" podStartSLOduration=7.678999086 podStartE2EDuration="8.714922674s" totalImagesPullingTime="949.074482ms" totalInitContainerRuntime="86.849106ms" isStatefulPod=true podCreationTimestamp="2026-07-23 19:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:36.713495436 +0000 UTC m=+72.002095029" watchObservedRunningTime="2026-07-23 19:17:36.714922674 +0000 UTC m=+72.003522266"
Jul 23 19:17:36.735532 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.735077    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-bgkmn" podStartSLOduration=3.390233437 podStartE2EDuration="8.735058483s" totalImagesPullingTime="5.344825046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:36.733441841 +0000 UTC m=+72.022041432" watchObservedRunningTime="2026-07-23 19:17:36.735058483 +0000 UTC m=+72.023658075"
Jul 23 19:17:36.805621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.805565    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qbd7m" podStartSLOduration=66.638916692 podStartE2EDuration="1m11.80554404s" totalImagesPullingTime="5.166627348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:36.787510372 +0000 UTC m=+72.076109962" watchObservedRunningTime="2026-07-23 19:17:36.80554404 +0000 UTC m=+72.094143631"
Jul 23 19:17:36.806791 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.805888    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-69v2n" podStartSLOduration=66.715876656 podStartE2EDuration="1m11.805875541s" totalImagesPullingTime="5.089998885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:36.805530033 +0000 UTC m=+72.094129623" watchObservedRunningTime="2026-07-23 19:17:36.805875541 +0000 UTC m=+72.094475135"
Jul 23 19:17:36.829026 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:36.828257    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5km4c" podStartSLOduration=65.15786748 podStartE2EDuration="1m10.828236957s" totalImagesPullingTime="5.670369477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:36.826880901 +0000 UTC m=+72.115480492" watchObservedRunningTime="2026-07-23 19:17:36.828236957 +0000 UTC m=+72.116836529"
Jul 23 19:17:42.704179 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.704075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb" event={"ID":"9b7569b2-3f22-4015-b8ad-9111642391b4","Type":"ContainerStarted","Data":"f84702b3ca49ecdfc39170530a081eca17c6682a0e79b97acf598785242867ad"}
Jul 23 19:17:42.706485 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.706431    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:42.720063 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.720026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9d57f47f-8gb2x" event={"ID":"c32f919d-9cca-434c-9cd0-4a1de158d12d","Type":"ContainerStarted","Data":"c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89"}
Jul 23 19:17:42.727134 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.726071    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb" podStartSLOduration=4.510313186 podStartE2EDuration="10.726050027s" totalImagesPullingTime="6.215736841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:42.72514583 +0000 UTC m=+78.013745422" watchObservedRunningTime="2026-07-23 19:17:42.726050027 +0000 UTC m=+78.014649620"
Jul 23 19:17:42.727696 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.727613    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwckb"
Jul 23 19:17:42.733583 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.733459    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"26ad706772632ef32d8229da982258ad507bff81f4454f7a11c6a4e1f7dfe2c9"}
Jul 23 19:17:42.733583 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.733529    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"b6b50c672c2b37f44c830ee19a581fb7a422db626304b8569890d29edd800ac5"}
Jul 23 19:17:42.736224 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.736155    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5" event={"ID":"fc141d97-e7fd-49f8-b878-5a7eea218ce2","Type":"ContainerStarted","Data":"fea859fb8676a0d134125a7b2de8e87625817e037975a6831a6c5f8583c1a99b"}
Jul 23 19:17:42.747989 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.747264    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"24652229fa4a5f1aef88b12b65e932badc46d3b4b3eaf249bf1f51df55524850"}
Jul 23 19:17:42.747989 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.747337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"a12a7ba80c06972cea19f196b94959a10abe9a7e8f5e5d017b1e2f80a385a5bc"}
Jul 23 19:17:42.761731 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.760886    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f9d57f47f-8gb2x" podStartSLOduration=4.54361794 podStartE2EDuration="10.760865567s" totalImagesPullingTime="6.217247627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:42.759964274 +0000 UTC m=+78.048563865" watchObservedRunningTime="2026-07-23 19:17:42.760865567 +0000 UTC m=+78.049465159"
Jul 23 19:17:42.763664 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.763639    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j" event={"ID":"f2ff0639-7b16-49d9-a210-2edce75e9233","Type":"ContainerStarted","Data":"1d563120def76d8f37828013c920eda3ee87cd00da91935024e37d9ef5220757"}
Jul 23 19:17:42.763848 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.763823    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j" event={"ID":"f2ff0639-7b16-49d9-a210-2edce75e9233","Type":"ContainerStarted","Data":"4534113b6577b27aafe47948d7091db51c5334164ec636cf30e147c954c048a9"}
Jul 23 19:17:42.775591 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.775442    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"2083fc5f81857cfab343ee9332a6e52472eebab3d36c48f593f228722a289394"}
Jul 23 19:17:42.775591 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.775520    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"c2c5c385e22c64c99cde9fe3f2bea165b1c6854a289f80bcced65498549e8cac"}
Jul 23 19:17:42.822052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:42.821232    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5" podStartSLOduration=4.50377894 podStartE2EDuration="10.821211066s" totalImagesPullingTime="6.317432126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:42.792456083 +0000 UTC m=+78.081055674" watchObservedRunningTime="2026-07-23 19:17:42.821211066 +0000 UTC m=+78.109810657"
Jul 23 19:17:43.240501 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.240471    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:43.240646 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.240526    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:43.242399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.242362    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:43.782797 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.782767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"c7571e5aae5e9d04a2711c4b0c2dc98dd3a16003e7a0560d64befbb790372f8f"}
Jul 23 19:17:43.783222 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.782827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"102dfae4d924ce85ba10127f8e9c620e8217d6ee5b71d6c938fc7da2430d802c"}
Jul 23 19:17:43.783222 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.782859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"53833c026b5bc02c3c06a45ed33b2e05b562bdbd8247062b1e7dc98c52d6ecdb"}
Jul 23 19:17:43.783222 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.782901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64a38aba-3f39-4bf7-85f0-da9ab233960e","Type":"ContainerStarted","Data":"e13df0555734b29b9ff60cbd3bcfce3469f57c24089f53f3ac36addff65199b1"}
Jul 23 19:17:43.784882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.784858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j" event={"ID":"f2ff0639-7b16-49d9-a210-2edce75e9233","Type":"ContainerStarted","Data":"d96b09f1b140f98fa145b8fc5ee080b56d632e0832aab175dc7ae90660180d91"}
Jul 23 19:17:43.787683 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.787645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"9f2a1c7ca78d1fbab7f7dad8331c9bdbfd31dc7a5bce529e4646937756766810"}
Jul 23 19:17:43.787790 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.787736    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"253eee15ccf51b8c564721f83d770dbec1b976d1b171f24bd38aa9164c22e083"}
Jul 23 19:17:43.787790 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.787782    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"94678845a65d49c0a697426e958d94ba3005843a12bd82ffeb9911b8b1609355"}
Jul 23 19:17:43.789617 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.789596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"a40fdf66dbf089601e4ecfd7c4ebe674d8ebb89e6eac2e34cf7e47602cb43ec2"}
Jul 23 19:17:43.793337 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.793318    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:17:43.851131 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.851079    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-757d74c77c-rk76j" podStartSLOduration=4.414723664 podStartE2EDuration="10.851063034s" totalImagesPullingTime="6.43633937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:43.847498979 +0000 UTC m=+79.136098570" watchObservedRunningTime="2026-07-23 19:17:43.851063034 +0000 UTC m=+79.139662625"
Jul 23 19:17:43.851435 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:43.851409    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.102147377 podStartE2EDuration="9.851400541s" totalImagesPullingTime="5.727485692s" totalInitContainerRuntime="21.767472ms" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:43.822863736 +0000 UTC m=+79.111463330" watchObservedRunningTime="2026-07-23 19:17:43.851400541 +0000 UTC m=+79.140000110"
Jul 23 19:17:44.776148 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.776108    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:17:44.798542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.798504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"1e66a17389f96c74167968c257c3abce374d06a0158502e64dea97cbb946f48f"}
Jul 23 19:17:44.799027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.798593    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"59e5b841531bb1019d65708b01d8ed82d15785095a2bf8b292ea9830f0cb1bd4"}
Jul 23 19:17:44.799027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.798633    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" event={"ID":"063fda30-5178-419b-9ef9-9e8d3d56fbb8","Type":"ContainerStarted","Data":"23744a837cb1fb2df2840dca523330cb31b0c235bb1876a7ad3e5bf6ca83a857"}
Jul 23 19:17:44.799445 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.799421    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:44.803369 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.802861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df8184d7-bfab-46ef-bc1e-8a9dda541093","Type":"ContainerStarted","Data":"4c64eaec1ee26acd432926857216c96d3a30681ebc54abc02953ef06cf9c1f94"}
Jul 23 19:17:44.826736 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.826658    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2" podStartSLOduration=5.8716709510000005 podStartE2EDuration="13.826637731s" totalImagesPullingTime="7.95496678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:44.823594847 +0000 UTC m=+80.112194439" watchObservedRunningTime="2026-07-23 19:17:44.826637731 +0000 UTC m=+80.115237323"
Jul 23 19:17:44.854734 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:44.854655    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.928859199 podStartE2EDuration="15.854637296s" totalImagesPullingTime="11.90045354s" totalInitContainerRuntime="25.324557ms" isStatefulPod=false podCreationTimestamp="2026-07-23 19:17:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:17:44.852741891 +0000 UTC m=+80.141341482" watchObservedRunningTime="2026-07-23 19:17:44.854637296 +0000 UTC m=+80.143236887"
Jul 23 19:17:51.817791 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:51.817648    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-cfcccfcf9-dg4l2"
Jul 23 19:17:53.041935 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:53.041900    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:53.042394 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:53.041991    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:17:56.059058 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:17:56.059026    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9d57f47f-8gb2x"]
Jul 23 19:18:08.682924 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:08.682894    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-69v2n"
Jul 23 19:18:13.047259 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:13.047229    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:18:13.051083 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:13.051055    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f495dc744-qx6s5"
Jul 23 19:18:21.080525 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.080461    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f9d57f47f-8gb2x" podUID="c32f919d-9cca-434c-9cd0-4a1de158d12d" containerName="console" containerID="cri-o://c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89" gracePeriod=15
Jul 23 19:18:21.357873 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.357851    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.440970    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-oauth-config\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.441027    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-serving-cert\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.441080    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-service-ca\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.441130    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-config\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.441199    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-trusted-ca-bundle\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.441238    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvzv2\" (UniqueName: \"kubernetes.io/projected/c32f919d-9cca-434c-9cd0-4a1de158d12d-kube-api-access-zvzv2\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.441268    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-oauth-serving-cert\") pod \"c32f919d-9cca-434c-9cd0-4a1de158d12d\" (UID: \"c32f919d-9cca-434c-9cd0-4a1de158d12d\") "
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.442163    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-service-ca" (OuterVolumeSpecName: "service-ca") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.442577    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.443029    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-config" (OuterVolumeSpecName: "console-config") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:18:21.446782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.443379    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:18:21.452024 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.451992    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c32f919d-9cca-434c-9cd0-4a1de158d12d-kube-api-access-zvzv2" (OuterVolumeSpecName: "kube-api-access-zvzv2") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "kube-api-access-zvzv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:18:21.452191 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.452163    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:18:21.453917 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.453901    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c32f919d-9cca-434c-9cd0-4a1de158d12d" (UID: "c32f919d-9cca-434c-9cd0-4a1de158d12d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542180    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvzv2\" (UniqueName: \"kubernetes.io/projected/c32f919d-9cca-434c-9cd0-4a1de158d12d-kube-api-access-zvzv2\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542218    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-oauth-serving-cert\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542235    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-oauth-config\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542250    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-serving-cert\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542265    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542280    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-console-config\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.542322 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.542294    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32f919d-9cca-434c-9cd0-4a1de158d12d-trusted-ca-bundle\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:18:21.971765 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.971724    2566 generic.go:356] "Generic (PLEG): container finished" podID="c32f919d-9cca-434c-9cd0-4a1de158d12d" containerID="c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89" exitCode=0
Jul 23 19:18:21.971992 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.971839    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9d57f47f-8gb2x" event={"ID":"c32f919d-9cca-434c-9cd0-4a1de158d12d","Type":"ContainerDied","Data":"c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89"}
Jul 23 19:18:21.971992 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.971891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9d57f47f-8gb2x" event={"ID":"c32f919d-9cca-434c-9cd0-4a1de158d12d","Type":"ContainerDied","Data":"d7874fd9b453d609b6cffbf523e28000045ad2b9bbf19b6322eeab556e1e36fb"}
Jul 23 19:18:21.971992 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.971922    2566 scope.go:117] "RemoveContainer" containerID="c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89"
Jul 23 19:18:21.972160 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.972085    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9d57f47f-8gb2x"
Jul 23 19:18:21.984393 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.984373    2566 scope.go:117] "RemoveContainer" containerID="c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89"
Jul 23 19:18:21.984663 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:18:21.984640    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89\": container with ID starting with c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89 not found: ID does not exist" containerID="c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89"
Jul 23 19:18:21.984777 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.984690    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89"} err="failed to get container status \"c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89\": rpc error: code = NotFound desc = could not find container \"c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89\": container with ID starting with c45a014a43659acb5b7e617ea1e911b0440f682105b4dae79580147a4ea3ad89 not found: ID does not exist"
Jul 23 19:18:21.997573 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:21.997547    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9d57f47f-8gb2x"]
Jul 23 19:18:22.001696 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:22.001657    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f9d57f47f-8gb2x"]
Jul 23 19:18:23.241454 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:23.241417    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c32f919d-9cca-434c-9cd0-4a1de158d12d" path="/var/lib/kubelet/pods/c32f919d-9cca-434c-9cd0-4a1de158d12d/volumes"
Jul 23 19:18:34.776129 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:34.776089    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:18:34.795811 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:34.795783    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:18:35.039631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:18:35.039552    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 19:20:22.647982 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.647944    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kppvh"]
Jul 23 19:20:22.648472 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.648319    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c32f919d-9cca-434c-9cd0-4a1de158d12d" containerName="console"
Jul 23 19:20:22.648472 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.648332    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c32f919d-9cca-434c-9cd0-4a1de158d12d" containerName="console"
Jul 23 19:20:22.648472 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.648396    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c32f919d-9cca-434c-9cd0-4a1de158d12d" containerName="console"
Jul 23 19:20:22.650377 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.650360    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.654213 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.654193    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sd4hk\""
Jul 23 19:20:22.654753 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.654738    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 19:20:22.654836 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.654778    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 19:20:22.684135 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.684093    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kppvh"]
Jul 23 19:20:22.721381 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.721343    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x52md\" (UniqueName: \"kubernetes.io/projected/85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353-kube-api-access-x52md\") pod \"cert-manager-webhook-6486ffd997-kppvh\" (UID: \"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.721541 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.721393    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kppvh\" (UID: \"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.822547 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.822513    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x52md\" (UniqueName: \"kubernetes.io/projected/85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353-kube-api-access-x52md\") pod \"cert-manager-webhook-6486ffd997-kppvh\" (UID: \"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.822783 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.822600    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kppvh\" (UID: \"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.833557 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.833526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x52md\" (UniqueName: \"kubernetes.io/projected/85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353-kube-api-access-x52md\") pod \"cert-manager-webhook-6486ffd997-kppvh\" (UID: \"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.839632 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.839610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kppvh\" (UID: \"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:22.970361 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:22.970325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:23.101115 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:23.101090    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kppvh"]
Jul 23 19:20:23.103358 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:20:23.103322    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85bcfb1f_ba6f_4c80_8522_2ea4eb8bc353.slice/crio-90fbe23302fe2d98f1c76f469917b41763b8e3a03ff1b561cfc995717d67388a WatchSource:0}: Error finding container 90fbe23302fe2d98f1c76f469917b41763b8e3a03ff1b561cfc995717d67388a: Status 404 returned error can't find the container with id 90fbe23302fe2d98f1c76f469917b41763b8e3a03ff1b561cfc995717d67388a
Jul 23 19:20:23.442588 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:23.442555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh" event={"ID":"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353","Type":"ContainerStarted","Data":"90fbe23302fe2d98f1c76f469917b41763b8e3a03ff1b561cfc995717d67388a"}
Jul 23 19:20:26.458495 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:26.458404    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh" event={"ID":"85bcfb1f-ba6f-4c80-8522-2ea4eb8bc353","Type":"ContainerStarted","Data":"a44a33734bc3585917cb70e8766ef181882002fafef11826b3e16706d8cd2e9f"}
Jul 23 19:20:26.458936 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:26.458546    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:32.464561 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:32.464530    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh"
Jul 23 19:20:32.486532 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:32.486473    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kppvh" podStartSLOduration=7.405478666 podStartE2EDuration="10.486455146s" totalImagesPullingTime="3.08097648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:20:26.492254101 +0000 UTC m=+241.780853692" watchObservedRunningTime="2026-07-23 19:20:32.486455146 +0000 UTC m=+247.775054739"
Jul 23 19:20:32.925917 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:32.925882    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wf64p"]
Jul 23 19:20:32.928187 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:32.928168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:32.930554 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:32.930535    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-d74b5\""
Jul 23 19:20:32.938245 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:32.938225    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wf64p"]
Jul 23 19:20:33.022343 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.022307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6tcv\" (UniqueName: \"kubernetes.io/projected/ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd-kube-api-access-r6tcv\") pod \"cert-manager-55b4d954d5-wf64p\" (UID: \"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd\") " pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.022343 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.022348    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd-bound-sa-token\") pod \"cert-manager-55b4d954d5-wf64p\" (UID: \"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd\") " pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.122801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.122767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd-bound-sa-token\") pod \"cert-manager-55b4d954d5-wf64p\" (UID: \"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd\") " pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.122948 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.122858    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6tcv\" (UniqueName: \"kubernetes.io/projected/ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd-kube-api-access-r6tcv\") pod \"cert-manager-55b4d954d5-wf64p\" (UID: \"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd\") " pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.141940 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.141915    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd-bound-sa-token\") pod \"cert-manager-55b4d954d5-wf64p\" (UID: \"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd\") " pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.142340 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.142323    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6tcv\" (UniqueName: \"kubernetes.io/projected/ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd-kube-api-access-r6tcv\") pod \"cert-manager-55b4d954d5-wf64p\" (UID: \"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd\") " pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.236572 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.236497    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wf64p"
Jul 23 19:20:33.367837 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.367802    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wf64p"]
Jul 23 19:20:33.370060 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:20:33.370030    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddcaa1bf_f5f2_45e4_a550_6d8dea7eb6cd.slice/crio-b867947bd50949fa0f707c3413e8d2ad3161df85cf320b313feb1f990f698bd5 WatchSource:0}: Error finding container b867947bd50949fa0f707c3413e8d2ad3161df85cf320b313feb1f990f698bd5: Status 404 returned error can't find the container with id b867947bd50949fa0f707c3413e8d2ad3161df85cf320b313feb1f990f698bd5
Jul 23 19:20:33.487585 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.487502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wf64p" event={"ID":"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd","Type":"ContainerStarted","Data":"c45685e2842b8015232c57c99bdfc8f10d19126e807ba1cacff059862e5c7d32"}
Jul 23 19:20:33.487585 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.487542    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wf64p" event={"ID":"ddcaa1bf-f5f2-45e4-a550-6d8dea7eb6cd","Type":"ContainerStarted","Data":"b867947bd50949fa0f707c3413e8d2ad3161df85cf320b313feb1f990f698bd5"}
Jul 23 19:20:33.507841 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:33.507795    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wf64p" podStartSLOduration=1.507781271 podStartE2EDuration="1.507781271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:20:33.505940693 +0000 UTC m=+248.794540285" watchObservedRunningTime="2026-07-23 19:20:33.507781271 +0000 UTC m=+248.796380861"
Jul 23 19:20:57.962751 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.962658    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"]
Jul 23 19:20:57.967123 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.967091    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:57.975560 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.975349    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 19:20:57.975560 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.975370    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 19:20:57.975560 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.975420    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hbs2v\""
Jul 23 19:20:57.975560 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.975375    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 19:20:57.980011 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:57.979994    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 19:20:58.000424 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.000400    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"]
Jul 23 19:20:58.058132 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.058095    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c533b160-80f6-48e1-8978-0d9d4c200aa8-tmp\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.058313 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.058139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c533b160-80f6-48e1-8978-0d9d4c200aa8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.058313 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.058231    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cmb6\" (UniqueName: \"kubernetes.io/projected/c533b160-80f6-48e1-8978-0d9d4c200aa8-kube-api-access-2cmb6\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.058313 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.058294    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c533b160-80f6-48e1-8978-0d9d4c200aa8-manifests\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.058438 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.058336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c533b160-80f6-48e1-8978-0d9d4c200aa8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.159102 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.159072    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cmb6\" (UniqueName: \"kubernetes.io/projected/c533b160-80f6-48e1-8978-0d9d4c200aa8-kube-api-access-2cmb6\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.159102 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.159110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c533b160-80f6-48e1-8978-0d9d4c200aa8-manifests\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.159315 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.159132    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c533b160-80f6-48e1-8978-0d9d4c200aa8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.159315 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.159250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c533b160-80f6-48e1-8978-0d9d4c200aa8-tmp\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.159315 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.159306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c533b160-80f6-48e1-8978-0d9d4c200aa8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.159519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.159497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c533b160-80f6-48e1-8978-0d9d4c200aa8-manifests\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.161516 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.161495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c533b160-80f6-48e1-8978-0d9d4c200aa8-tmp\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.161893 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.161873    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c533b160-80f6-48e1-8978-0d9d4c200aa8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.161966 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.161874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c533b160-80f6-48e1-8978-0d9d4c200aa8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.170255 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.170232    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cmb6\" (UniqueName: \"kubernetes.io/projected/c533b160-80f6-48e1-8978-0d9d4c200aa8-kube-api-access-2cmb6\") pod \"opendatahub-operator-controller-manager-6bf4f8c896-7hpfq\" (UID: \"c533b160-80f6-48e1-8978-0d9d4c200aa8\") " pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.279423 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.279340    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:20:58.431660 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.431636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"]
Jul 23 19:20:58.434214 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:20:58.434188    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc533b160_80f6_48e1_8978_0d9d4c200aa8.slice/crio-fdee45b855922b3b01a69bee7c7fd932f49cb3e53b834266ddef9b4834b4e9b1 WatchSource:0}: Error finding container fdee45b855922b3b01a69bee7c7fd932f49cb3e53b834266ddef9b4834b4e9b1: Status 404 returned error can't find the container with id fdee45b855922b3b01a69bee7c7fd932f49cb3e53b834266ddef9b4834b4e9b1
Jul 23 19:20:58.589268 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:20:58.589177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq" event={"ID":"c533b160-80f6-48e1-8978-0d9d4c200aa8","Type":"ContainerStarted","Data":"fdee45b855922b3b01a69bee7c7fd932f49cb3e53b834266ddef9b4834b4e9b1"}
Jul 23 19:21:02.617068 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:02.617027    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq" event={"ID":"c533b160-80f6-48e1-8978-0d9d4c200aa8","Type":"ContainerStarted","Data":"60b633b2c74b5472ea4798da369e4c72832dd3d58d7c7922177a612c91cf7ec8"}
Jul 23 19:21:03.622561 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:03.622470    2566 generic.go:356] "Generic (PLEG): container finished" podID="c533b160-80f6-48e1-8978-0d9d4c200aa8" containerID="60b633b2c74b5472ea4798da369e4c72832dd3d58d7c7922177a612c91cf7ec8" exitCode=0
Jul 23 19:21:03.622561 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:03.622531    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq" event={"ID":"c533b160-80f6-48e1-8978-0d9d4c200aa8","Type":"ContainerDied","Data":"60b633b2c74b5472ea4798da369e4c72832dd3d58d7c7922177a612c91cf7ec8"}
Jul 23 19:21:04.629968 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:04.629933    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq" event={"ID":"c533b160-80f6-48e1-8978-0d9d4c200aa8","Type":"ContainerStarted","Data":"39f502636fb679b195b077e02ddd06beed85d1fc70ee4a602e200216ea369449"}
Jul 23 19:21:04.630385 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:04.630064    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:21:04.667582 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:04.667532    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq" podStartSLOduration=2.4200762989999998 podStartE2EDuration="7.667514145s" totalImagesPullingTime="4.173354021s" totalInitContainerRuntime="1.074083825s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:20:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:21:04.665553615 +0000 UTC m=+279.954153206" watchObservedRunningTime="2026-07-23 19:21:04.667514145 +0000 UTC m=+279.956113735"
Jul 23 19:21:15.637150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:15.637110    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bf4f8c896-7hpfq"
Jul 23 19:21:17.466655 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.466617    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"]
Jul 23 19:21:17.470041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.470020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.472621 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.472597    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 19:21:17.472762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.472621    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 19:21:17.473564 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.473530    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-89hwp\""
Jul 23 19:21:17.473702 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.473612    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 19:21:17.473702 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.473686    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 19:21:17.473801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.473740    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 19:21:17.493103 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.493074    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"]
Jul 23 19:21:17.635995 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.635955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-metrics-cert\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.636183 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.636021    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-cert\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.636183 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.636104    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-manager-config\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.636275 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.636253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlsz8\" (UniqueName: \"kubernetes.io/projected/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-kube-api-access-wlsz8\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.737756 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.737653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlsz8\" (UniqueName: \"kubernetes.io/projected/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-kube-api-access-wlsz8\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.737921 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.737781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-metrics-cert\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.737921 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.737837    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-cert\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.737921 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.737870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-manager-config\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.738506 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.738484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-manager-config\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.740416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.740394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-metrics-cert\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.740523 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.740462    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-cert\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.750743 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.750718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlsz8\" (UniqueName: \"kubernetes.io/projected/c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca-kube-api-access-wlsz8\") pod \"lws-controller-manager-6fcc6d8b99-nx2w2\" (UID: \"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca\") " pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.779865 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.779838    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:17.937133 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:17.937103    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"]
Jul 23 19:21:17.943945 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:21:17.943915    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73b206c_bb1d_4cb8_88e9_07f69ba9f9ca.slice/crio-c925b3cc86124d97640165f4fc64428d39449a4ffd34796b0645d6087163d143 WatchSource:0}: Error finding container c925b3cc86124d97640165f4fc64428d39449a4ffd34796b0645d6087163d143: Status 404 returned error can't find the container with id c925b3cc86124d97640165f4fc64428d39449a4ffd34796b0645d6087163d143
Jul 23 19:21:18.687053 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.687021    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"]
Jul 23 19:21:18.692028 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.692003    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.692838 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.692807    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2" event={"ID":"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca","Type":"ContainerStarted","Data":"c925b3cc86124d97640165f4fc64428d39449a4ffd34796b0645d6087163d143"}
Jul 23 19:21:18.700700 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.700501    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 19:21:18.701061 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.701044    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pcf2s\""
Jul 23 19:21:18.701687 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.701140    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 19:21:18.702345 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.701959    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 19:21:18.702345 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.702197    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 19:21:18.737273 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.737235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"]
Jul 23 19:21:18.847819 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.847776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d05744-7bc2-4479-928d-4d8e18a829ec-tls-certs\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.847986 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.847911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89d05744-7bc2-4479-928d-4d8e18a829ec-tmp\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.847986 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.847957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj4w7\" (UniqueName: \"kubernetes.io/projected/89d05744-7bc2-4479-928d-4d8e18a829ec-kube-api-access-vj4w7\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.949231 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.949135    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89d05744-7bc2-4479-928d-4d8e18a829ec-tmp\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.949397 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.949232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj4w7\" (UniqueName: \"kubernetes.io/projected/89d05744-7bc2-4479-928d-4d8e18a829ec-kube-api-access-vj4w7\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.949397 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.949367    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d05744-7bc2-4479-928d-4d8e18a829ec-tls-certs\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.952307 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.952269    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89d05744-7bc2-4479-928d-4d8e18a829ec-tmp\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.952437 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.952357    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d05744-7bc2-4479-928d-4d8e18a829ec-tls-certs\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:18.959090 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:18.959065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj4w7\" (UniqueName: \"kubernetes.io/projected/89d05744-7bc2-4479-928d-4d8e18a829ec-kube-api-access-vj4w7\") pod \"kube-auth-proxy-6474cf8df7-nb642\" (UID: \"89d05744-7bc2-4479-928d-4d8e18a829ec\") " pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:19.005185 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:19.005155    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"
Jul 23 19:21:19.169703 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:19.169660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6474cf8df7-nb642"]
Jul 23 19:21:19.171383 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:21:19.171360    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d05744_7bc2_4479_928d_4d8e18a829ec.slice/crio-955ccd8290b99671085dd2de025550296c3bfc4112b1b606a0a902e85ce30483 WatchSource:0}: Error finding container 955ccd8290b99671085dd2de025550296c3bfc4112b1b606a0a902e85ce30483: Status 404 returned error can't find the container with id 955ccd8290b99671085dd2de025550296c3bfc4112b1b606a0a902e85ce30483
Jul 23 19:21:19.698041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:19.698008    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642" event={"ID":"89d05744-7bc2-4479-928d-4d8e18a829ec","Type":"ContainerStarted","Data":"955ccd8290b99671085dd2de025550296c3bfc4112b1b606a0a902e85ce30483"}
Jul 23 19:21:22.714388 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:22.714350    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642" event={"ID":"89d05744-7bc2-4479-928d-4d8e18a829ec","Type":"ContainerStarted","Data":"2175b44e703f39d8e2baa3a85cd4a2c38c9dd68a34226ebd0b94797b54ec6960"}
Jul 23 19:21:22.740615 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:22.740557    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6474cf8df7-nb642" podStartSLOduration=1.884614705 podStartE2EDuration="4.740539577s" totalImagesPullingTime="2.855924872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:21:22.739059119 +0000 UTC m=+298.027658713" watchObservedRunningTime="2026-07-23 19:21:22.740539577 +0000 UTC m=+298.029139167"
Jul 23 19:21:25.086309 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:25.086283    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 19:21:31.192068 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.192036    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"]
Jul 23 19:21:31.201867 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.201843    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.210914 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.210894    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 19:21:31.211392 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.211380    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zkvqs\""
Jul 23 19:21:31.215945 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.215920    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"]
Jul 23 19:21:31.269423 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.269377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377dc391-fe18-445f-9435-064f1a6f47b2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w4xps\" (UID: \"377dc391-fe18-445f-9435-064f1a6f47b2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.269611 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.269561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7nln\" (UniqueName: \"kubernetes.io/projected/377dc391-fe18-445f-9435-064f1a6f47b2-kube-api-access-g7nln\") pod \"llmisvc-controller-manager-777c6bd7d5-w4xps\" (UID: \"377dc391-fe18-445f-9435-064f1a6f47b2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.371043 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.370952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7nln\" (UniqueName: \"kubernetes.io/projected/377dc391-fe18-445f-9435-064f1a6f47b2-kube-api-access-g7nln\") pod \"llmisvc-controller-manager-777c6bd7d5-w4xps\" (UID: \"377dc391-fe18-445f-9435-064f1a6f47b2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.371191 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.371082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377dc391-fe18-445f-9435-064f1a6f47b2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w4xps\" (UID: \"377dc391-fe18-445f-9435-064f1a6f47b2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.373706 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.373684    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377dc391-fe18-445f-9435-064f1a6f47b2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w4xps\" (UID: \"377dc391-fe18-445f-9435-064f1a6f47b2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.397198 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.397166    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7nln\" (UniqueName: \"kubernetes.io/projected/377dc391-fe18-445f-9435-064f1a6f47b2-kube-api-access-g7nln\") pod \"llmisvc-controller-manager-777c6bd7d5-w4xps\" (UID: \"377dc391-fe18-445f-9435-064f1a6f47b2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.511782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.511743    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:31.734919 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.734887    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"]
Jul 23 19:21:31.745521 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:21:31.745490    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod377dc391_fe18_445f_9435_064f1a6f47b2.slice/crio-6483b5e759f29d687ada91a887a0322eb336d39b52ad7d78149c2233d9072dd6 WatchSource:0}: Error finding container 6483b5e759f29d687ada91a887a0322eb336d39b52ad7d78149c2233d9072dd6: Status 404 returned error can't find the container with id 6483b5e759f29d687ada91a887a0322eb336d39b52ad7d78149c2233d9072dd6
Jul 23 19:21:31.746967 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.746948    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:21:31.758115 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.758092    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2" event={"ID":"c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca","Type":"ContainerStarted","Data":"31a26ad045888c022b508edd7a029bc17b01cb849de3d8309ceb417e4036dada"}
Jul 23 19:21:31.758225 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.758202    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:21:31.759242 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.759221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps" event={"ID":"377dc391-fe18-445f-9435-064f1a6f47b2","Type":"ContainerStarted","Data":"6483b5e759f29d687ada91a887a0322eb336d39b52ad7d78149c2233d9072dd6"}
Jul 23 19:21:31.805267 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:31.805217    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2" podStartSLOduration=1.6739059539999999 podStartE2EDuration="14.805201342s" totalImagesPullingTime="13.131295388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:21:31.804362866 +0000 UTC m=+307.092962456" watchObservedRunningTime="2026-07-23 19:21:31.805201342 +0000 UTC m=+307.093800932"
Jul 23 19:21:36.785800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:36.785768    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps" event={"ID":"377dc391-fe18-445f-9435-064f1a6f47b2","Type":"ContainerStarted","Data":"1e8dc1f8dbb10c13468286a794ac827748c090113522f9d5bb1ab234db0adf69"}
Jul 23 19:21:36.786183 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:36.785892    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:21:36.807541 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:36.807488    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps" podStartSLOduration=1.878352714 podStartE2EDuration="5.807472394s" totalImagesPullingTime="3.92911968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:21:36.805666537 +0000 UTC m=+312.094266129" watchObservedRunningTime="2026-07-23 19:21:36.807472394 +0000 UTC m=+312.096071985"
Jul 23 19:21:42.767173 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:21:42.767143    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6fcc6d8b99-nx2w2"
Jul 23 19:22:07.793252 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:22:07.793218    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w4xps"
Jul 23 19:23:14.732472 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.732432    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"]
Jul 23 19:23:14.734942 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.734926    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:14.737590 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.737569    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-shfjv\""
Jul 23 19:23:14.740601 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.740580    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 19:23:14.741919 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.741900    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 19:23:14.759313 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.759292    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"]
Jul 23 19:23:14.793658 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.793630    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w6nb\" (UniqueName: \"kubernetes.io/projected/e7043bb5-0c55-40c3-a322-6f36f156f35e-kube-api-access-4w6nb\") pod \"kuadrant-operator-controller-manager-86878c769f-6flvv\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:14.793790 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.793691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7043bb5-0c55-40c3-a322-6f36f156f35e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6flvv\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:14.894738 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.894696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w6nb\" (UniqueName: \"kubernetes.io/projected/e7043bb5-0c55-40c3-a322-6f36f156f35e-kube-api-access-4w6nb\") pod \"kuadrant-operator-controller-manager-86878c769f-6flvv\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:14.894933 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.894760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7043bb5-0c55-40c3-a322-6f36f156f35e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6flvv\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:14.895134 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.895114    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7043bb5-0c55-40c3-a322-6f36f156f35e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6flvv\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:14.909325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:14.909296    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w6nb\" (UniqueName: \"kubernetes.io/projected/e7043bb5-0c55-40c3-a322-6f36f156f35e-kube-api-access-4w6nb\") pod \"kuadrant-operator-controller-manager-86878c769f-6flvv\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:15.045157 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:15.045070    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:15.187599 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:15.187567    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"]
Jul 23 19:23:15.191126 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:23:15.191099    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7043bb5_0c55_40c3_a322_6f36f156f35e.slice/crio-d4a28079189d5467e6ed47da22d39da47e4917f73de0e4c199502022e84dd213 WatchSource:0}: Error finding container d4a28079189d5467e6ed47da22d39da47e4917f73de0e4c199502022e84dd213: Status 404 returned error can't find the container with id d4a28079189d5467e6ed47da22d39da47e4917f73de0e4c199502022e84dd213
Jul 23 19:23:15.219914 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:15.219877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv" event={"ID":"e7043bb5-0c55-40c3-a322-6f36f156f35e","Type":"ContainerStarted","Data":"d4a28079189d5467e6ed47da22d39da47e4917f73de0e4c199502022e84dd213"}
Jul 23 19:23:20.248935 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:20.248907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv" event={"ID":"e7043bb5-0c55-40c3-a322-6f36f156f35e","Type":"ContainerStarted","Data":"b3964284f9b9a1d487675e0db6b574617fcd0ae922e664c63c318b772047d557"}
Jul 23 19:23:20.249296 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:20.249012    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:20.271009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:20.270955    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv" podStartSLOduration=1.283309987 podStartE2EDuration="6.270935499s" totalImagesPullingTime="4.987625512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:23:20.269958435 +0000 UTC m=+415.558558025" watchObservedRunningTime="2026-07-23 19:23:20.270935499 +0000 UTC m=+415.559535092"
Jul 23 19:23:31.255823 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:31.255779    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:23:51.355094 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.355007    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rhdll"]
Jul 23 19:23:51.357645 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.357623    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:23:51.360291 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.360269    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sgfl8\""
Jul 23 19:23:51.366965 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.366940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rhdll"]
Jul 23 19:23:51.436663 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.436628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ttcb\" (UniqueName: \"kubernetes.io/projected/3bbcf1b2-69da-4d17-9388-c138187787f4-kube-api-access-4ttcb\") pod \"authorino-75d556bd77-rhdll\" (UID: \"3bbcf1b2-69da-4d17-9388-c138187787f4\") " pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:23:51.537801 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.537774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ttcb\" (UniqueName: \"kubernetes.io/projected/3bbcf1b2-69da-4d17-9388-c138187787f4-kube-api-access-4ttcb\") pod \"authorino-75d556bd77-rhdll\" (UID: \"3bbcf1b2-69da-4d17-9388-c138187787f4\") " pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:23:51.546541 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.546513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ttcb\" (UniqueName: \"kubernetes.io/projected/3bbcf1b2-69da-4d17-9388-c138187787f4-kube-api-access-4ttcb\") pod \"authorino-75d556bd77-rhdll\" (UID: \"3bbcf1b2-69da-4d17-9388-c138187787f4\") " pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:23:51.667633 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.667588    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:23:51.795423 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:51.795392    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rhdll"]
Jul 23 19:23:51.798474 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:23:51.798447    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bbcf1b2_69da_4d17_9388_c138187787f4.slice/crio-242fe69142472be0a3892e8b2e2ace133cec18e9ec17c174c787ffc942f2972f WatchSource:0}: Error finding container 242fe69142472be0a3892e8b2e2ace133cec18e9ec17c174c787ffc942f2972f: Status 404 returned error can't find the container with id 242fe69142472be0a3892e8b2e2ace133cec18e9ec17c174c787ffc942f2972f
Jul 23 19:23:52.408969 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:52.408929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rhdll" event={"ID":"3bbcf1b2-69da-4d17-9388-c138187787f4","Type":"ContainerStarted","Data":"242fe69142472be0a3892e8b2e2ace133cec18e9ec17c174c787ffc942f2972f"}
Jul 23 19:23:55.432743 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:55.432706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rhdll" event={"ID":"3bbcf1b2-69da-4d17-9388-c138187787f4","Type":"ContainerStarted","Data":"ffceddbff1a9b2fa7fe6204f0866e7346f101401308b789774451c5c8b12915a"}
Jul 23 19:23:55.448605 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:23:55.448560    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rhdll" podStartSLOduration=1.375443131 podStartE2EDuration="4.448542546s" totalImagesPullingTime="3.073099415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:23:55.44835171 +0000 UTC m=+450.736951301" watchObservedRunningTime="2026-07-23 19:23:55.448542546 +0000 UTC m=+450.737142139"
Jul 23 19:24:27.348018 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.347981    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rhdll"]
Jul 23 19:24:27.348522 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.348171    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rhdll" podUID="3bbcf1b2-69da-4d17-9388-c138187787f4" containerName="authorino" containerID="cri-o://ffceddbff1a9b2fa7fe6204f0866e7346f101401308b789774451c5c8b12915a" gracePeriod=30
Jul 23 19:24:27.584299 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.584257    2566 generic.go:356] "Generic (PLEG): container finished" podID="3bbcf1b2-69da-4d17-9388-c138187787f4" containerID="ffceddbff1a9b2fa7fe6204f0866e7346f101401308b789774451c5c8b12915a" exitCode=0
Jul 23 19:24:27.584408 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.584297    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rhdll" event={"ID":"3bbcf1b2-69da-4d17-9388-c138187787f4","Type":"ContainerDied","Data":"ffceddbff1a9b2fa7fe6204f0866e7346f101401308b789774451c5c8b12915a"}
Jul 23 19:24:27.584408 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.584344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rhdll" event={"ID":"3bbcf1b2-69da-4d17-9388-c138187787f4","Type":"ContainerDied","Data":"242fe69142472be0a3892e8b2e2ace133cec18e9ec17c174c787ffc942f2972f"}
Jul 23 19:24:27.584408 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.584355    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="242fe69142472be0a3892e8b2e2ace133cec18e9ec17c174c787ffc942f2972f"
Jul 23 19:24:27.592787 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.592768    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:24:27.656573 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.656547    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ttcb\" (UniqueName: \"kubernetes.io/projected/3bbcf1b2-69da-4d17-9388-c138187787f4-kube-api-access-4ttcb\") pod \"3bbcf1b2-69da-4d17-9388-c138187787f4\" (UID: \"3bbcf1b2-69da-4d17-9388-c138187787f4\") "
Jul 23 19:24:27.658858 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.658826    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bbcf1b2-69da-4d17-9388-c138187787f4-kube-api-access-4ttcb" (OuterVolumeSpecName: "kube-api-access-4ttcb") pod "3bbcf1b2-69da-4d17-9388-c138187787f4" (UID: "3bbcf1b2-69da-4d17-9388-c138187787f4"). InnerVolumeSpecName "kube-api-access-4ttcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:24:27.758043 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:27.758006    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ttcb\" (UniqueName: \"kubernetes.io/projected/3bbcf1b2-69da-4d17-9388-c138187787f4-kube-api-access-4ttcb\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:24:28.589121 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:28.589091    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rhdll"
Jul 23 19:24:28.609970 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:28.609946    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rhdll"]
Jul 23 19:24:28.616489 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:28.616464    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rhdll"]
Jul 23 19:24:29.238594 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:29.238553    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bbcf1b2-69da-4d17-9388-c138187787f4" path="/var/lib/kubelet/pods/3bbcf1b2-69da-4d17-9388-c138187787f4/volumes"
Jul 23 19:24:32.068891 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.068861    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b9fbb96bd-kgrw2"]
Jul 23 19:24:32.069391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.069371    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bbcf1b2-69da-4d17-9388-c138187787f4" containerName="authorino"
Jul 23 19:24:32.069465 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.069395    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bbcf1b2-69da-4d17-9388-c138187787f4" containerName="authorino"
Jul 23 19:24:32.069519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.069500    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bbcf1b2-69da-4d17-9388-c138187787f4" containerName="authorino"
Jul 23 19:24:32.071721 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.071702    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.074177 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.074155    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mfldh\""
Jul 23 19:24:32.074905 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.074888    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 19:24:32.075735 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.075718    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 19:24:32.084694 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.084655    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b9fbb96bd-kgrw2"]
Jul 23 19:24:32.192774 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.192742    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpnfz\" (UniqueName: \"kubernetes.io/projected/013411a0-d882-4123-af66-5bc15b320eea-kube-api-access-mpnfz\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.192932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.192799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.293755 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.293719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpnfz\" (UniqueName: \"kubernetes.io/projected/013411a0-d882-4123-af66-5bc15b320eea-kube-api-access-mpnfz\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.293908 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.293769    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.293954 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:24:32.293912    2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 23 19:24:32.293985 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:24:32.293971    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert podName:013411a0-d882-4123-af66-5bc15b320eea nodeName:}" failed. No retries permitted until 2026-07-23 19:24:32.793955393 +0000 UTC m=+488.082554961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert") pod "maas-controller-b9fbb96bd-kgrw2" (UID: "013411a0-d882-4123-af66-5bc15b320eea") : secret "maas-controller-webhook-cert" not found
Jul 23 19:24:32.305095 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.305072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpnfz\" (UniqueName: \"kubernetes.io/projected/013411a0-d882-4123-af66-5bc15b320eea-kube-api-access-mpnfz\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.798052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:32.798023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:32.798224 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:24:32.798135    2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 23 19:24:32.798224 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:24:32.798204    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert podName:013411a0-d882-4123-af66-5bc15b320eea nodeName:}" failed. No retries permitted until 2026-07-23 19:24:33.798184305 +0000 UTC m=+489.086783876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert") pod "maas-controller-b9fbb96bd-kgrw2" (UID: "013411a0-d882-4123-af66-5bc15b320eea") : secret "maas-controller-webhook-cert" not found
Jul 23 19:24:33.806887 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:33.806851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:33.809334 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:33.809315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") pod \"maas-controller-b9fbb96bd-kgrw2\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") " pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:33.882401 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:33.882363    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:34.011785 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:34.011762    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b9fbb96bd-kgrw2"]
Jul 23 19:24:34.014405 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:24:34.014377    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013411a0_d882_4123_af66_5bc15b320eea.slice/crio-357b18b809de240c8440b03a0eef5aa94cf47e8d30fd8d77c89e84b0ec61e6b8 WatchSource:0}: Error finding container 357b18b809de240c8440b03a0eef5aa94cf47e8d30fd8d77c89e84b0ec61e6b8: Status 404 returned error can't find the container with id 357b18b809de240c8440b03a0eef5aa94cf47e8d30fd8d77c89e84b0ec61e6b8
Jul 23 19:24:34.621226 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:34.621182    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2" event={"ID":"013411a0-d882-4123-af66-5bc15b320eea","Type":"ContainerStarted","Data":"357b18b809de240c8440b03a0eef5aa94cf47e8d30fd8d77c89e84b0ec61e6b8"}
Jul 23 19:24:36.633483 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:36.633396    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2" event={"ID":"013411a0-d882-4123-af66-5bc15b320eea","Type":"ContainerStarted","Data":"3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11"}
Jul 23 19:24:36.633855 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:36.633537    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:36.655064 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:36.655016    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2" podStartSLOduration=2.467165495 podStartE2EDuration="4.655004071s" totalImagesPullingTime="2.187838576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:24:36.652619041 +0000 UTC m=+491.941218646" watchObservedRunningTime="2026-07-23 19:24:36.655004071 +0000 UTC m=+491.943603662"
Jul 23 19:24:47.409056 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.409020    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b9fbb96bd-kgrw2"]
Jul 23 19:24:47.409435 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.409237    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2" podUID="013411a0-d882-4123-af66-5bc15b320eea" containerName="manager" containerID="cri-o://3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11" gracePeriod=30
Jul 23 19:24:47.411221 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.411198    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:47.677707 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.677666    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:47.688093 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.688072    2566 generic.go:356] "Generic (PLEG): container finished" podID="013411a0-d882-4123-af66-5bc15b320eea" containerID="3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11" exitCode=0
Jul 23 19:24:47.688182 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.688115    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2"
Jul 23 19:24:47.688182 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.688153    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2" event={"ID":"013411a0-d882-4123-af66-5bc15b320eea","Type":"ContainerDied","Data":"3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11"}
Jul 23 19:24:47.688251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.688200    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b9fbb96bd-kgrw2" event={"ID":"013411a0-d882-4123-af66-5bc15b320eea","Type":"ContainerDied","Data":"357b18b809de240c8440b03a0eef5aa94cf47e8d30fd8d77c89e84b0ec61e6b8"}
Jul 23 19:24:47.688251 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.688221    2566 scope.go:117] "RemoveContainer" containerID="3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11"
Jul 23 19:24:47.696298 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.696276    2566 scope.go:117] "RemoveContainer" containerID="3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11"
Jul 23 19:24:47.696615 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:24:47.696587    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11\": container with ID starting with 3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11 not found: ID does not exist" containerID="3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11"
Jul 23 19:24:47.696714 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.696626    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11"} err="failed to get container status \"3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11\": rpc error: code = NotFound desc = could not find container \"3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11\": container with ID starting with 3432964def8a2fade19e5e358a6fe1248ba2db08cead93c104743c2e518fdd11 not found: ID does not exist"
Jul 23 19:24:47.831112 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.831071    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpnfz\" (UniqueName: \"kubernetes.io/projected/013411a0-d882-4123-af66-5bc15b320eea-kube-api-access-mpnfz\") pod \"013411a0-d882-4123-af66-5bc15b320eea\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") "
Jul 23 19:24:47.831288 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.831127    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") pod \"013411a0-d882-4123-af66-5bc15b320eea\" (UID: \"013411a0-d882-4123-af66-5bc15b320eea\") "
Jul 23 19:24:47.833354 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.833324    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "013411a0-d882-4123-af66-5bc15b320eea" (UID: "013411a0-d882-4123-af66-5bc15b320eea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:24:47.833427 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.833366    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013411a0-d882-4123-af66-5bc15b320eea-kube-api-access-mpnfz" (OuterVolumeSpecName: "kube-api-access-mpnfz") pod "013411a0-d882-4123-af66-5bc15b320eea" (UID: "013411a0-d882-4123-af66-5bc15b320eea"). InnerVolumeSpecName "kube-api-access-mpnfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:24:47.932754 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.932623    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpnfz\" (UniqueName: \"kubernetes.io/projected/013411a0-d882-4123-af66-5bc15b320eea-kube-api-access-mpnfz\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:24:47.932754 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:47.932698    2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/013411a0-d882-4123-af66-5bc15b320eea-webhook-cert\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:24:48.011790 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:48.011763    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b9fbb96bd-kgrw2"]
Jul 23 19:24:48.017307 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:48.017283    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b9fbb96bd-kgrw2"]
Jul 23 19:24:49.237866 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:24:49.237827    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="013411a0-d882-4123-af66-5bc15b320eea" path="/var/lib/kubelet/pods/013411a0-d882-4123-af66-5bc15b320eea/volumes"
Jul 23 19:25:09.213366 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.213325    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wz468"]
Jul 23 19:25:09.213987 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.213786    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="013411a0-d882-4123-af66-5bc15b320eea" containerName="manager"
Jul 23 19:25:09.213987 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.213800    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="013411a0-d882-4123-af66-5bc15b320eea" containerName="manager"
Jul 23 19:25:09.213987 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.213883    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="013411a0-d882-4123-af66-5bc15b320eea" containerName="manager"
Jul 23 19:25:09.221620 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.221599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.224555 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.224528    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 23 19:25:09.224710 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.224621    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bdv66\""
Jul 23 19:25:09.228665 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.228638    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wz468"]
Jul 23 19:25:09.310160 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.310123    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xns\" (UniqueName: \"kubernetes.io/projected/7dd161f0-1d45-467f-b0e6-f513745ba9d9-kube-api-access-g6xns\") pod \"payload-processing-5bf47cd988-wz468\" (UID: \"7dd161f0-1d45-467f-b0e6-f513745ba9d9\") " pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.310313 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.310185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd161f0-1d45-467f-b0e6-f513745ba9d9-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wz468\" (UID: \"7dd161f0-1d45-467f-b0e6-f513745ba9d9\") " pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.411622 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.411584    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xns\" (UniqueName: \"kubernetes.io/projected/7dd161f0-1d45-467f-b0e6-f513745ba9d9-kube-api-access-g6xns\") pod \"payload-processing-5bf47cd988-wz468\" (UID: \"7dd161f0-1d45-467f-b0e6-f513745ba9d9\") " pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.411802 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.411652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd161f0-1d45-467f-b0e6-f513745ba9d9-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wz468\" (UID: \"7dd161f0-1d45-467f-b0e6-f513745ba9d9\") " pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.412339 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.412321    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd161f0-1d45-467f-b0e6-f513745ba9d9-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wz468\" (UID: \"7dd161f0-1d45-467f-b0e6-f513745ba9d9\") " pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.423005 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.422969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xns\" (UniqueName: \"kubernetes.io/projected/7dd161f0-1d45-467f-b0e6-f513745ba9d9-kube-api-access-g6xns\") pod \"payload-processing-5bf47cd988-wz468\" (UID: \"7dd161f0-1d45-467f-b0e6-f513745ba9d9\") " pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.534395 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.534311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:09.673556 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.673526    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wz468"]
Jul 23 19:25:09.675494 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:25:09.675468    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd161f0_1d45_467f_b0e6_f513745ba9d9.slice/crio-78e3cc272f73dac3367a60f1fb836199ad4611678412a03d3e88efc62e42eb2e WatchSource:0}: Error finding container 78e3cc272f73dac3367a60f1fb836199ad4611678412a03d3e88efc62e42eb2e: Status 404 returned error can't find the container with id 78e3cc272f73dac3367a60f1fb836199ad4611678412a03d3e88efc62e42eb2e
Jul 23 19:25:09.797796 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:09.797717    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wz468" event={"ID":"7dd161f0-1d45-467f-b0e6-f513745ba9d9","Type":"ContainerStarted","Data":"78e3cc272f73dac3367a60f1fb836199ad4611678412a03d3e88efc62e42eb2e"}
Jul 23 19:25:11.816580 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:11.816543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wz468" event={"ID":"7dd161f0-1d45-467f-b0e6-f513745ba9d9","Type":"ContainerStarted","Data":"9bbb84038116adc7bd9b90e330f4ab7b856fa445f20cba9934fbb5edcb630052"}
Jul 23 19:25:11.817082 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:11.816716    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:11.833282 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:11.833227    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-wz468" podStartSLOduration=1.328337659 podStartE2EDuration="2.83321266s" totalImagesPullingTime="1.504875001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:25:11.831106907 +0000 UTC m=+527.119706490" watchObservedRunningTime="2026-07-23 19:25:11.83321266 +0000 UTC m=+527.121812252"
Jul 23 19:25:22.823374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:22.823344    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-wz468"
Jul 23 19:25:24.382337 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.382296    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"]
Jul 23 19:25:24.385918 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.385896    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.388318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.388284    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6h2fl\""
Jul 23 19:25:24.388318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.388305    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 19:25:24.388497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.388435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 19:25:24.389188 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.389168    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 23 19:25:24.401278 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.401253    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"]
Jul 23 19:25:24.459287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.459255    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.459287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.459291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8dn6\" (UniqueName: \"kubernetes.io/projected/8b220048-3e22-4d30-8321-2714a6a74dc9-kube-api-access-f8dn6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.459519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.459326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.459519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.459346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.459519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.459379    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b220048-3e22-4d30-8321-2714a6a74dc9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.459519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.459475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561059 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561059 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561062    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561297 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8dn6\" (UniqueName: \"kubernetes.io/projected/8b220048-3e22-4d30-8321-2714a6a74dc9-kube-api-access-f8dn6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561297 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561297 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561129    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561297 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b220048-3e22-4d30-8321-2714a6a74dc9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561555 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561532    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561598 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.561659 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.561626    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.563552 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.563534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b220048-3e22-4d30-8321-2714a6a74dc9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.563820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.563803    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b220048-3e22-4d30-8321-2714a6a74dc9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.569491 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.569465    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8dn6\" (UniqueName: \"kubernetes.io/projected/8b220048-3e22-4d30-8321-2714a6a74dc9-kube-api-access-f8dn6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww\" (UID: \"8b220048-3e22-4d30-8321-2714a6a74dc9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.696103 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.696074    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:24.836595 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.836511    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"]
Jul 23 19:25:24.838981 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:25:24.838953    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b220048_3e22_4d30_8321_2714a6a74dc9.slice/crio-f5322f3bab2621a8603e6d2d80ec8a59d77cc3781b497bb8f7b071c602b84d2b WatchSource:0}: Error finding container f5322f3bab2621a8603e6d2d80ec8a59d77cc3781b497bb8f7b071c602b84d2b: Status 404 returned error can't find the container with id f5322f3bab2621a8603e6d2d80ec8a59d77cc3781b497bb8f7b071c602b84d2b
Jul 23 19:25:24.879365 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:24.879332    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww" event={"ID":"8b220048-3e22-4d30-8321-2714a6a74dc9","Type":"ContainerStarted","Data":"f5322f3bab2621a8603e6d2d80ec8a59d77cc3781b497bb8f7b071c602b84d2b"}
Jul 23 19:25:26.890700 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:26.890647    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww" event={"ID":"8b220048-3e22-4d30-8321-2714a6a74dc9","Type":"ContainerStarted","Data":"39d7941979fe8235f4ac8029c38c343071b14c7fe8976f0c5a4ad39622006e93"}
Jul 23 19:25:26.912275 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:26.912227    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww" podStartSLOduration=1.863643323 podStartE2EDuration="2.912212169s" totalImagesPullingTime="1.048568846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:25:26.911084723 +0000 UTC m=+542.199684315" watchObservedRunningTime="2026-07-23 19:25:26.912212169 +0000 UTC m=+542.200811758"
Jul 23 19:25:34.696391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:34.696350    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:34.696391 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:34.696404    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:34.709344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:34.709318    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:34.945405 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:34.945377    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww"
Jul 23 19:25:50.467736 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.467701    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"]
Jul 23 19:25:50.471072 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.471056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.473987 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.473965    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 23 19:25:50.481846 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.481824    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"]
Jul 23 19:25:50.494989 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.494960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4rh\" (UniqueName: \"kubernetes.io/projected/246d10e9-4866-4b65-bfaa-686277c73c03-kube-api-access-jz4rh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.495113 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.495012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.495113 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.495081    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.495216 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.495140    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.495216 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.495196    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246d10e9-4866-4b65-bfaa-686277c73c03-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.495297 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.495256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.596562 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.596525    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.596747 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.596572    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4rh\" (UniqueName: \"kubernetes.io/projected/246d10e9-4866-4b65-bfaa-686277c73c03-kube-api-access-jz4rh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.596747 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.596724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.596827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.596785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.596860 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.596849    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.596948 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.596926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246d10e9-4866-4b65-bfaa-686277c73c03-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.597104 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.597080    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.597168 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.597129    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.597168 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.597160    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.598999 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.598974    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246d10e9-4866-4b65-bfaa-686277c73c03-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.599376 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.599358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246d10e9-4866-4b65-bfaa-686277c73c03-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.604616 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.604596    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4rh\" (UniqueName: \"kubernetes.io/projected/246d10e9-4866-4b65-bfaa-686277c73c03-kube-api-access-jz4rh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg\" (UID: \"246d10e9-4866-4b65-bfaa-686277c73c03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.781920 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.781835    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:25:50.916358 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:50.916330    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"]
Jul 23 19:25:50.918637 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:25:50.918611    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246d10e9_4866_4b65_bfaa_686277c73c03.slice/crio-57f477b25c80f99758db921fa91fae62241d3fbfb3457163a2ffa7efcafc62f9 WatchSource:0}: Error finding container 57f477b25c80f99758db921fa91fae62241d3fbfb3457163a2ffa7efcafc62f9: Status 404 returned error can't find the container with id 57f477b25c80f99758db921fa91fae62241d3fbfb3457163a2ffa7efcafc62f9
Jul 23 19:25:51.018648 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:51.018604    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg" event={"ID":"246d10e9-4866-4b65-bfaa-686277c73c03","Type":"ContainerStarted","Data":"57f477b25c80f99758db921fa91fae62241d3fbfb3457163a2ffa7efcafc62f9"}
Jul 23 19:25:52.025262 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:52.025221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg" event={"ID":"246d10e9-4866-4b65-bfaa-686277c73c03","Type":"ContainerStarted","Data":"08d6f6c6415c65e80758f7071718d4da452799ffecbf79311b7864bf2f6b8599"}
Jul 23 19:25:52.043731 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:52.043657    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg" podStartSLOduration=1.852294944 podStartE2EDuration="2.043641316s" totalImagesPullingTime="191.346372ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:25:52.041479016 +0000 UTC m=+567.330078609" watchObservedRunningTime="2026-07-23 19:25:52.043641316 +0000 UTC m=+567.332240907"
Jul 23 19:25:57.966858 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:57.966825    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"]
Jul 23 19:25:57.970715 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:57.970695    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:57.973050 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:57.973031    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 23 19:25:57.981624 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:57.981597    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"]
Jul 23 19:25:58.067250 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.067215    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.067414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.067257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b61ba91-7eeb-4301-8077-ddb7091f726d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.067414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.067278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.067414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.067391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54jf\" (UniqueName: \"kubernetes.io/projected/6b61ba91-7eeb-4301-8077-ddb7091f726d-kube-api-access-h54jf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.067515 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.067419    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.067515 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.067453    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168197 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54jf\" (UniqueName: \"kubernetes.io/projected/6b61ba91-7eeb-4301-8077-ddb7091f726d-kube-api-access-h54jf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168275    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168334    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b61ba91-7eeb-4301-8077-ddb7091f726d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168748 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168841 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168760    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.168841 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.168788    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.170798 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.170779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b61ba91-7eeb-4301-8077-ddb7091f726d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.171058 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.171038    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b61ba91-7eeb-4301-8077-ddb7091f726d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.177065 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.177045    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54jf\" (UniqueName: \"kubernetes.io/projected/6b61ba91-7eeb-4301-8077-ddb7091f726d-kube-api-access-h54jf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sc2pd\" (UID: \"6b61ba91-7eeb-4301-8077-ddb7091f726d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.283524 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.283438    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:25:58.421927 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:58.421892    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"]
Jul 23 19:25:58.425212 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:25:58.425184    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b61ba91_7eeb_4301_8077_ddb7091f726d.slice/crio-77beb3b6e4b0aea5f4e188a410e0a079451438ba2a41f583e08a59b0bae8d1f8 WatchSource:0}: Error finding container 77beb3b6e4b0aea5f4e188a410e0a079451438ba2a41f583e08a59b0bae8d1f8: Status 404 returned error can't find the container with id 77beb3b6e4b0aea5f4e188a410e0a079451438ba2a41f583e08a59b0bae8d1f8
Jul 23 19:25:59.071532 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:59.071489    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd" event={"ID":"6b61ba91-7eeb-4301-8077-ddb7091f726d","Type":"ContainerStarted","Data":"25683a49a731ebe23ffe51bd87e1deb6e4a9f8cd005a0a26cb1ae3c134a4069f"}
Jul 23 19:25:59.071934 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:59.071540    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd" event={"ID":"6b61ba91-7eeb-4301-8077-ddb7091f726d","Type":"ContainerStarted","Data":"77beb3b6e4b0aea5f4e188a410e0a079451438ba2a41f583e08a59b0bae8d1f8"}
Jul 23 19:25:59.091202 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:25:59.091156    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd" podStartSLOduration=1.891309005 podStartE2EDuration="2.091141895s" totalImagesPullingTime="199.83289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:25:59.089054893 +0000 UTC m=+574.377654484" watchObservedRunningTime="2026-07-23 19:25:59.091141895 +0000 UTC m=+574.379741486"
Jul 23 19:26:00.782579 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:00.782545    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:26:00.783029 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:00.782604    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:26:00.795816 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:00.795792    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:26:01.092413 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:01.092322    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg"
Jul 23 19:26:06.783401 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.783344    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"]
Jul 23 19:26:06.794990 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.794965    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.797339 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.797312    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"]
Jul 23 19:26:06.797549 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.797528    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 19:26:06.850776 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.850738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24vdq\" (UniqueName: \"kubernetes.io/projected/74e72d08-6041-4ed2-bcda-a583049acd4d-kube-api-access-24vdq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.850925 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.850798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.850925 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.850914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e72d08-6041-4ed2-bcda-a583049acd4d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.851009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.850968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.851009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.850984    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.851009 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.850998    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952077 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24vdq\" (UniqueName: \"kubernetes.io/projected/74e72d08-6041-4ed2-bcda-a583049acd4d-kube-api-access-24vdq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952226 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952093    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952226 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952175    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e72d08-6041-4ed2-bcda-a583049acd4d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952351 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952227    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952351 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952251    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952351 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952599 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952599 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952595    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.952852 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.952638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.954588 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.954567    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e72d08-6041-4ed2-bcda-a583049acd4d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.954907 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.954889    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e72d08-6041-4ed2-bcda-a583049acd4d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:06.960561 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:06.960541    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24vdq\" (UniqueName: \"kubernetes.io/projected/74e72d08-6041-4ed2-bcda-a583049acd4d-kube-api-access-24vdq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d\" (UID: \"74e72d08-6041-4ed2-bcda-a583049acd4d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:07.105490 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:07.105404    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:07.243509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:07.243457    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"]
Jul 23 19:26:07.245891 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:26:07.245863    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74e72d08_6041_4ed2_bcda_a583049acd4d.slice/crio-1e209338a1bb40f72c6f6fb0d85efec444bf0f5d210f5c5209f02109f760e4f4 WatchSource:0}: Error finding container 1e209338a1bb40f72c6f6fb0d85efec444bf0f5d210f5c5209f02109f760e4f4: Status 404 returned error can't find the container with id 1e209338a1bb40f72c6f6fb0d85efec444bf0f5d210f5c5209f02109f760e4f4
Jul 23 19:26:08.116109 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:08.116070    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d" event={"ID":"74e72d08-6041-4ed2-bcda-a583049acd4d","Type":"ContainerStarted","Data":"d16f731309ab9b629c3666c3c3c94d4b09c94720014b016991ab6831b5995ec1"}
Jul 23 19:26:08.116519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:08.116117    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d" event={"ID":"74e72d08-6041-4ed2-bcda-a583049acd4d","Type":"ContainerStarted","Data":"1e209338a1bb40f72c6f6fb0d85efec444bf0f5d210f5c5209f02109f760e4f4"}
Jul 23 19:26:08.134629 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:08.134583    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d" podStartSLOduration=1.948364691 podStartE2EDuration="2.134566986s" totalImagesPullingTime="186.202295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:26:08.133714654 +0000 UTC m=+583.422314245" watchObservedRunningTime="2026-07-23 19:26:08.134566986 +0000 UTC m=+583.423166576"
Jul 23 19:26:08.283911 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:08.283878    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:26:08.284078 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:08.283933    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:26:08.296353 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:08.296328    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:26:09.133842 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:09.133806    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sc2pd"
Jul 23 19:26:17.105759 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:17.105730    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:17.105759 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:17.105765    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:17.118335 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:17.118309    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:26:17.172821 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:26:17.172792    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d"
Jul 23 19:30:25.145920 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:30:25.145885    2566 scope.go:117] "RemoveContainer" containerID="ffceddbff1a9b2fa7fe6204f0866e7346f101401308b789774451c5c8b12915a"
Jul 23 19:34:22.899245 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:22.899209    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"]
Jul 23 19:34:22.899713 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:22.899457    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv" podUID="e7043bb5-0c55-40c3-a322-6f36f156f35e" containerName="manager" containerID="cri-o://b3964284f9b9a1d487675e0db6b574617fcd0ae922e664c63c318b772047d557" gracePeriod=10
Jul 23 19:34:26.566783 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:26.566748    2566 generic.go:356] "Generic (PLEG): container finished" podID="e7043bb5-0c55-40c3-a322-6f36f156f35e" containerID="b3964284f9b9a1d487675e0db6b574617fcd0ae922e664c63c318b772047d557" exitCode=0
Jul 23 19:34:26.567153 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:26.566794    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv" event={"ID":"e7043bb5-0c55-40c3-a322-6f36f156f35e","Type":"ContainerDied","Data":"b3964284f9b9a1d487675e0db6b574617fcd0ae922e664c63c318b772047d557"}
Jul 23 19:34:27.152535 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.152510    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:34:27.250349 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.250323    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7043bb5-0c55-40c3-a322-6f36f156f35e-extensions-socket-volume\") pod \"e7043bb5-0c55-40c3-a322-6f36f156f35e\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") "
Jul 23 19:34:27.250496 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.250410    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w6nb\" (UniqueName: \"kubernetes.io/projected/e7043bb5-0c55-40c3-a322-6f36f156f35e-kube-api-access-4w6nb\") pod \"e7043bb5-0c55-40c3-a322-6f36f156f35e\" (UID: \"e7043bb5-0c55-40c3-a322-6f36f156f35e\") "
Jul 23 19:34:27.250765 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.250738    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7043bb5-0c55-40c3-a322-6f36f156f35e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7043bb5-0c55-40c3-a322-6f36f156f35e" (UID: "e7043bb5-0c55-40c3-a322-6f36f156f35e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:34:27.252521 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.252490    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7043bb5-0c55-40c3-a322-6f36f156f35e-kube-api-access-4w6nb" (OuterVolumeSpecName: "kube-api-access-4w6nb") pod "e7043bb5-0c55-40c3-a322-6f36f156f35e" (UID: "e7043bb5-0c55-40c3-a322-6f36f156f35e"). InnerVolumeSpecName "kube-api-access-4w6nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:34:27.351878 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.351808    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7043bb5-0c55-40c3-a322-6f36f156f35e-extensions-socket-volume\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:34:27.351878 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.351832    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w6nb\" (UniqueName: \"kubernetes.io/projected/e7043bb5-0c55-40c3-a322-6f36f156f35e-kube-api-access-4w6nb\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:34:27.573609 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.573581    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"
Jul 23 19:34:27.574027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.573618    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv" event={"ID":"e7043bb5-0c55-40c3-a322-6f36f156f35e","Type":"ContainerDied","Data":"d4a28079189d5467e6ed47da22d39da47e4917f73de0e4c199502022e84dd213"}
Jul 23 19:34:27.574027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.573703    2566 scope.go:117] "RemoveContainer" containerID="b3964284f9b9a1d487675e0db6b574617fcd0ae922e664c63c318b772047d557"
Jul 23 19:34:27.594847 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.594823    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"]
Jul 23 19:34:27.599193 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:27.599173    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6flvv"]
Jul 23 19:34:29.239281 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:34:29.239242    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7043bb5-0c55-40c3-a322-6f36f156f35e" path="/var/lib/kubelet/pods/e7043bb5-0c55-40c3-a322-6f36f156f35e/volumes"
Jul 23 19:38:02.087501 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.087472    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"]
Jul 23 19:38:02.087965 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.087916    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7043bb5-0c55-40c3-a322-6f36f156f35e" containerName="manager"
Jul 23 19:38:02.087965 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.087930    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7043bb5-0c55-40c3-a322-6f36f156f35e" containerName="manager"
Jul 23 19:38:02.088035 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.088020    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7043bb5-0c55-40c3-a322-6f36f156f35e" containerName="manager"
Jul 23 19:38:02.091353 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.091327    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.094150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.094124    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-1acc3c2e-kserve-self-signed-certs\""
Jul 23 19:38:02.109544 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.109517    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"]
Jul 23 19:38:02.182888 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.182850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-tmp-dir\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.183052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.182896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-model-cache\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.183052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.182985    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021feadd-dd03-468f-b903-cb88e109c79e-tls-certs\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.183126 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.183053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-home\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.183126 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.183074    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-dshm\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.183188 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.183127    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvjkb\" (UniqueName: \"kubernetes.io/projected/021feadd-dd03-468f-b903-cb88e109c79e-kube-api-access-lvjkb\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284174 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-model-cache\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021feadd-dd03-468f-b903-cb88e109c79e-tls-certs\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-home\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-dshm\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284541 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvjkb\" (UniqueName: \"kubernetes.io/projected/021feadd-dd03-468f-b903-cb88e109c79e-kube-api-access-lvjkb\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284541 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-tmp-dir\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284655 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284589    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-model-cache\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284756 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-home\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.284816 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.284752    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-tmp-dir\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.286808 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.286784    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-dshm\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.286916 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.286894    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021feadd-dd03-468f-b903-cb88e109c79e-tls-certs\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.293347 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.293327    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvjkb\" (UniqueName: \"kubernetes.io/projected/021feadd-dd03-468f-b903-cb88e109c79e-kube-api-access-lvjkb\") pod \"e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") " pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.401171 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.401144    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:02.538956 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.538925    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"]
Jul 23 19:38:02.542079 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:38:02.542050    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod021feadd_dd03_468f_b903_cb88e109c79e.slice/crio-6c341cf576609f2dfe9dfe63e4234a90cd89f9f1ff924da80822f33a92388bf1 WatchSource:0}: Error finding container 6c341cf576609f2dfe9dfe63e4234a90cd89f9f1ff924da80822f33a92388bf1: Status 404 returned error can't find the container with id 6c341cf576609f2dfe9dfe63e4234a90cd89f9f1ff924da80822f33a92388bf1
Jul 23 19:38:02.598500 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:02.598473    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p" event={"ID":"021feadd-dd03-468f-b903-cb88e109c79e","Type":"ContainerStarted","Data":"6c341cf576609f2dfe9dfe63e4234a90cd89f9f1ff924da80822f33a92388bf1"}
Jul 23 19:38:03.604448 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:03.604413    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p" event={"ID":"021feadd-dd03-468f-b903-cb88e109c79e","Type":"ContainerStarted","Data":"7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5"}
Jul 23 19:38:03.622756 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:03.622706    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p" podStartSLOduration=1.622690309 podStartE2EDuration="1.622690309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:38:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:38:03.621411829 +0000 UTC m=+1298.910011415" watchObservedRunningTime="2026-07-23 19:38:03.622690309 +0000 UTC m=+1298.911289894"
Jul 23 19:38:12.402142 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:12.402104    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:12.402578 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:12.402152    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:12.415585 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:12.415561    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:12.658891 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:12.658810    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:38:19.349882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.349847    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"]
Jul 23 19:38:19.353774 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.353658    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.356767 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.356746    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-1acc3c2e-kserve-self-signed-certs\""
Jul 23 19:38:19.379994 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.379962    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"]
Jul 23 19:38:19.452143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.452105    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-dshm\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.452143 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.452146    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-model-cache\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.452348 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.452245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tls-certs\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.452348 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.452273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62778\" (UniqueName: \"kubernetes.io/projected/585c55f3-d405-4b10-b290-ab0d6c67e4f1-kube-api-access-62778\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.452348 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.452302    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tmp-dir\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.452447 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.452392    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-home\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553140 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-dshm\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553140 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553143    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-model-cache\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tls-certs\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62778\" (UniqueName: \"kubernetes.io/projected/585c55f3-d405-4b10-b290-ab0d6c67e4f1-kube-api-access-62778\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tmp-dir\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553344 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553324    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-home\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553638 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553609    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-model-cache\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553638 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553631    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-home\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.553870 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.553705    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tmp-dir\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.555602 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.555580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-dshm\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.555799 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.555782    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tls-certs\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.561773 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.561756    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62778\" (UniqueName: \"kubernetes.io/projected/585c55f3-d405-4b10-b290-ab0d6c67e4f1-kube-api-access-62778\") pod \"e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") " pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.666716 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.666653    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:19.806856 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:19.806831    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"]
Jul 23 19:38:19.809341 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:38:19.809316    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod585c55f3_d405_4b10_b290_ab0d6c67e4f1.slice/crio-a2349f832a0fc7967192217c0b98f57df15ba205d82ece95fef260f15ec1b960 WatchSource:0}: Error finding container a2349f832a0fc7967192217c0b98f57df15ba205d82ece95fef260f15ec1b960: Status 404 returned error can't find the container with id a2349f832a0fc7967192217c0b98f57df15ba205d82ece95fef260f15ec1b960
Jul 23 19:38:20.689319 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:20.689281    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2" event={"ID":"585c55f3-d405-4b10-b290-ab0d6c67e4f1","Type":"ContainerStarted","Data":"a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc"}
Jul 23 19:38:20.689727 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:20.689327    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2" event={"ID":"585c55f3-d405-4b10-b290-ab0d6c67e4f1","Type":"ContainerStarted","Data":"a2349f832a0fc7967192217c0b98f57df15ba205d82ece95fef260f15ec1b960"}
Jul 23 19:38:20.706458 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:20.706405    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2" podStartSLOduration=1.706390217 podStartE2EDuration="1.706390217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:38:20.705324166 +0000 UTC m=+1315.993923756" watchObservedRunningTime="2026-07-23 19:38:20.706390217 +0000 UTC m=+1315.994989808"
Jul 23 19:38:29.667274 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:29.667238    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:29.667739 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:29.667290    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:29.679708 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:29.679665    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:29.744458 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:29.744423    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:38:55.266909 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:55.266875    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"]
Jul 23 19:38:55.983444 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:38:55.983408    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"]
Jul 23 19:39:10.288199 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.288137    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p" podUID="021feadd-dd03-468f-b903-cb88e109c79e" containerName="main" containerID="cri-o://7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5" gracePeriod=45
Jul 23 19:39:10.540273 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.540209    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:39:10.662494 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662455    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-model-cache\") pod \"021feadd-dd03-468f-b903-cb88e109c79e\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") "
Jul 23 19:39:10.662752 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662507    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-home\") pod \"021feadd-dd03-468f-b903-cb88e109c79e\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") "
Jul 23 19:39:10.662752 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662539    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvjkb\" (UniqueName: \"kubernetes.io/projected/021feadd-dd03-468f-b903-cb88e109c79e-kube-api-access-lvjkb\") pod \"021feadd-dd03-468f-b903-cb88e109c79e\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") "
Jul 23 19:39:10.662752 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662568    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-dshm\") pod \"021feadd-dd03-468f-b903-cb88e109c79e\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") "
Jul 23 19:39:10.662752 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662593    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021feadd-dd03-468f-b903-cb88e109c79e-tls-certs\") pod \"021feadd-dd03-468f-b903-cb88e109c79e\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") "
Jul 23 19:39:10.662752 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662618    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-tmp-dir\") pod \"021feadd-dd03-468f-b903-cb88e109c79e\" (UID: \"021feadd-dd03-468f-b903-cb88e109c79e\") "
Jul 23 19:39:10.662996 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662856    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-home" (OuterVolumeSpecName: "home") pod "021feadd-dd03-468f-b903-cb88e109c79e" (UID: "021feadd-dd03-468f-b903-cb88e109c79e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:10.662996 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.662883    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-model-cache" (OuterVolumeSpecName: "model-cache") pod "021feadd-dd03-468f-b903-cb88e109c79e" (UID: "021feadd-dd03-468f-b903-cb88e109c79e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:10.663097 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.663047    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "021feadd-dd03-468f-b903-cb88e109c79e" (UID: "021feadd-dd03-468f-b903-cb88e109c79e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:10.665073 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.665049    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-dshm" (OuterVolumeSpecName: "dshm") pod "021feadd-dd03-468f-b903-cb88e109c79e" (UID: "021feadd-dd03-468f-b903-cb88e109c79e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:10.665193 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.665086    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021feadd-dd03-468f-b903-cb88e109c79e-kube-api-access-lvjkb" (OuterVolumeSpecName: "kube-api-access-lvjkb") pod "021feadd-dd03-468f-b903-cb88e109c79e" (UID: "021feadd-dd03-468f-b903-cb88e109c79e"). InnerVolumeSpecName "kube-api-access-lvjkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:39:10.665193 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.665098    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021feadd-dd03-468f-b903-cb88e109c79e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "021feadd-dd03-468f-b903-cb88e109c79e" (UID: "021feadd-dd03-468f-b903-cb88e109c79e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:39:10.763614 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.763578    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-home\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.763614 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.763606    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvjkb\" (UniqueName: \"kubernetes.io/projected/021feadd-dd03-468f-b903-cb88e109c79e-kube-api-access-lvjkb\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.763614 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.763617    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-dshm\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.763851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.763625    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021feadd-dd03-468f-b903-cb88e109c79e-tls-certs\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.763851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.763634    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-tmp-dir\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.763851 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.763642    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/021feadd-dd03-468f-b903-cb88e109c79e-model-cache\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:10.934271 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.934234    2566 generic.go:356] "Generic (PLEG): container finished" podID="021feadd-dd03-468f-b903-cb88e109c79e" containerID="7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5" exitCode=0
Jul 23 19:39:10.934457 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.934286    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"
Jul 23 19:39:10.934457 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.934322    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p" event={"ID":"021feadd-dd03-468f-b903-cb88e109c79e","Type":"ContainerDied","Data":"7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5"}
Jul 23 19:39:10.934457 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.934366    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p" event={"ID":"021feadd-dd03-468f-b903-cb88e109c79e","Type":"ContainerDied","Data":"6c341cf576609f2dfe9dfe63e4234a90cd89f9f1ff924da80822f33a92388bf1"}
Jul 23 19:39:10.934457 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.934384    2566 scope.go:117] "RemoveContainer" containerID="7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5"
Jul 23 19:39:10.943665 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.943642    2566 scope.go:117] "RemoveContainer" containerID="7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5"
Jul 23 19:39:10.943967 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:39:10.943951    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5\": container with ID starting with 7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5 not found: ID does not exist" containerID="7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5"
Jul 23 19:39:10.944026 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.943976    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5"} err="failed to get container status \"7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5\": rpc error: code = NotFound desc = could not find container \"7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5\": container with ID starting with 7ea7207552fd49ad27bdb0d34dd326ee157f692c354462e37fdc78f16174f9a5 not found: ID does not exist"
Jul 23 19:39:10.955975 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.955952    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"]
Jul 23 19:39:10.959923 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:10.959905    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-1acc3c2e-kserve-6ccb446575-zrz2p"]
Jul 23 19:39:11.004109 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.004056    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2" podUID="585c55f3-d405-4b10-b290-ab0d6c67e4f1" containerName="main" containerID="cri-o://a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc" gracePeriod=45
Jul 23 19:39:11.239324 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.239294    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="021feadd-dd03-468f-b903-cb88e109c79e" path="/var/lib/kubelet/pods/021feadd-dd03-468f-b903-cb88e109c79e/volumes"
Jul 23 19:39:11.240357 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.240339    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:39:11.368933 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.368903    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62778\" (UniqueName: \"kubernetes.io/projected/585c55f3-d405-4b10-b290-ab0d6c67e4f1-kube-api-access-62778\") pod \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") "
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.368945    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tmp-dir\") pod \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") "
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.368981    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-home\") pod \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") "
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369048    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-dshm\") pod \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") "
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369132    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tls-certs\") pod \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") "
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369179    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-model-cache\") pod \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\" (UID: \"585c55f3-d405-4b10-b290-ab0d6c67e4f1\") "
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369248    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "585c55f3-d405-4b10-b290-ab0d6c67e4f1" (UID: "585c55f3-d405-4b10-b290-ab0d6c67e4f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:11.369325 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369305    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-home" (OuterVolumeSpecName: "home") pod "585c55f3-d405-4b10-b290-ab0d6c67e4f1" (UID: "585c55f3-d405-4b10-b290-ab0d6c67e4f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:11.369635 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369498    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-home\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:11.369635 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369515    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tmp-dir\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:11.369635 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.369495    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "585c55f3-d405-4b10-b290-ab0d6c67e4f1" (UID: "585c55f3-d405-4b10-b290-ab0d6c67e4f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:11.371246 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.371224    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585c55f3-d405-4b10-b290-ab0d6c67e4f1-kube-api-access-62778" (OuterVolumeSpecName: "kube-api-access-62778") pod "585c55f3-d405-4b10-b290-ab0d6c67e4f1" (UID: "585c55f3-d405-4b10-b290-ab0d6c67e4f1"). InnerVolumeSpecName "kube-api-access-62778". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:39:11.371775 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.371759    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "585c55f3-d405-4b10-b290-ab0d6c67e4f1" (UID: "585c55f3-d405-4b10-b290-ab0d6c67e4f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:39:11.371842 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.371770    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-dshm" (OuterVolumeSpecName: "dshm") pod "585c55f3-d405-4b10-b290-ab0d6c67e4f1" (UID: "585c55f3-d405-4b10-b290-ab0d6c67e4f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:39:11.470975 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.470938    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-dshm\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:11.470975 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.470967    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/585c55f3-d405-4b10-b290-ab0d6c67e4f1-tls-certs\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:11.470975 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.470977    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/585c55f3-d405-4b10-b290-ab0d6c67e4f1-model-cache\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:11.470975 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.470987    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62778\" (UniqueName: \"kubernetes.io/projected/585c55f3-d405-4b10-b290-ab0d6c67e4f1-kube-api-access-62778\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:39:11.941176 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.941139    2566 generic.go:356] "Generic (PLEG): container finished" podID="585c55f3-d405-4b10-b290-ab0d6c67e4f1" containerID="a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc" exitCode=0
Jul 23 19:39:11.941176 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.941181    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"
Jul 23 19:39:11.941383 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.941194    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2" event={"ID":"585c55f3-d405-4b10-b290-ab0d6c67e4f1","Type":"ContainerDied","Data":"a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc"}
Jul 23 19:39:11.941383 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.941231    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2" event={"ID":"585c55f3-d405-4b10-b290-ab0d6c67e4f1","Type":"ContainerDied","Data":"a2349f832a0fc7967192217c0b98f57df15ba205d82ece95fef260f15ec1b960"}
Jul 23 19:39:11.941383 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.941255    2566 scope.go:117] "RemoveContainer" containerID="a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc"
Jul 23 19:39:11.950685 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.950649    2566 scope.go:117] "RemoveContainer" containerID="a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc"
Jul 23 19:39:11.950949 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:39:11.950928    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc\": container with ID starting with a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc not found: ID does not exist" containerID="a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc"
Jul 23 19:39:11.950998 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.950959    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc"} err="failed to get container status \"a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc\": rpc error: code = NotFound desc = could not find container \"a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc\": container with ID starting with a9983a980683a3c219c7018e40af572c9e408b114cc361fb905f4a3c5fb063cc not found: ID does not exist"
Jul 23 19:39:11.961908 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.961883    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"]
Jul 23 19:39:11.964901 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:11.964878    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-1acc3c2e-kserve-6bdf59fbf5-jpjr2"]
Jul 23 19:39:13.238785 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:39:13.238754    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585c55f3-d405-4b10-b290-ab0d6c67e4f1" path="/var/lib/kubelet/pods/585c55f3-d405-4b10-b290-ab0d6c67e4f1/volumes"
Jul 23 19:42:41.610705 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.610643    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"]
Jul 23 19:42:41.611150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.611108    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021feadd-dd03-468f-b903-cb88e109c79e" containerName="main"
Jul 23 19:42:41.611150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.611123    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="021feadd-dd03-468f-b903-cb88e109c79e" containerName="main"
Jul 23 19:42:41.611150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.611134    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585c55f3-d405-4b10-b290-ab0d6c67e4f1" containerName="main"
Jul 23 19:42:41.611150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.611139    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="585c55f3-d405-4b10-b290-ab0d6c67e4f1" containerName="main"
Jul 23 19:42:41.611276 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.611224    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="585c55f3-d405-4b10-b290-ab0d6c67e4f1" containerName="main"
Jul 23 19:42:41.611276 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.611234    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="021feadd-dd03-468f-b903-cb88e109c79e" containerName="main"
Jul 23 19:42:41.614406 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.614391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.617066 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.617043    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:42:41.617230 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.617212    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:42:41.617936 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.617913    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:42:41.618026 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.617935    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c543a96f-serving-cert\""
Jul 23 19:42:41.626232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.626196    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"]
Jul 23 19:42:41.655904 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.655873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.656035 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.655926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h98q8\" (UniqueName: \"kubernetes.io/projected/130adbd9-2e38-4017-a401-77ba0906e8ef-kube-api-access-h98q8\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.656035 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.656006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130adbd9-2e38-4017-a401-77ba0906e8ef-openshift-service-ca\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.757158 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.757123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h98q8\" (UniqueName: \"kubernetes.io/projected/130adbd9-2e38-4017-a401-77ba0906e8ef-kube-api-access-h98q8\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.757309 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.757172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130adbd9-2e38-4017-a401-77ba0906e8ef-openshift-service-ca\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.757309 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.757226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.757379 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:42:41.757350    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-serving-cert: secret "maas-api-e2e-ait-c543a96f-serving-cert" not found
Jul 23 19:42:41.757437 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:42:41.757426    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls podName:130adbd9-2e38-4017-a401-77ba0906e8ef nodeName:}" failed. No retries permitted until 2026-07-23 19:42:42.257406014 +0000 UTC m=+1577.546005585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls") pod "maas-api-e2e-ait-c543a96f-765df777df-b56fh" (UID: "130adbd9-2e38-4017-a401-77ba0906e8ef") : secret "maas-api-e2e-ait-c543a96f-serving-cert" not found
Jul 23 19:42:41.757884 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.757865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130adbd9-2e38-4017-a401-77ba0906e8ef-openshift-service-ca\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:41.768352 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:41.768325    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h98q8\" (UniqueName: \"kubernetes.io/projected/130adbd9-2e38-4017-a401-77ba0906e8ef-kube-api-access-h98q8\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:42.262357 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:42.262317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:42.264921 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:42.264901    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls\") pod \"maas-api-e2e-ait-c543a96f-765df777df-b56fh\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:42.525495 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:42.525422    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:42.665977 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:42.665949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"]
Jul 23 19:42:42.668293 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:42:42.668268    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130adbd9_2e38_4017_a401_77ba0906e8ef.slice/crio-9b6c941445c126a4c13282dcca0d2d2f7dd61b452a1bcf647973eebf2bc4a2de WatchSource:0}: Error finding container 9b6c941445c126a4c13282dcca0d2d2f7dd61b452a1bcf647973eebf2bc4a2de: Status 404 returned error can't find the container with id 9b6c941445c126a4c13282dcca0d2d2f7dd61b452a1bcf647973eebf2bc4a2de
Jul 23 19:42:42.669986 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:42.669964    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:42:42.946013 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:42.945974    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh" event={"ID":"130adbd9-2e38-4017-a401-77ba0906e8ef","Type":"ContainerStarted","Data":"9b6c941445c126a4c13282dcca0d2d2f7dd61b452a1bcf647973eebf2bc4a2de"}
Jul 23 19:42:44.956454 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:44.956425    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh" event={"ID":"130adbd9-2e38-4017-a401-77ba0906e8ef","Type":"ContainerStarted","Data":"28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3"}
Jul 23 19:42:44.956846 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:44.956553    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:42:44.973084 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:44.973033    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh" podStartSLOduration=2.130645539 podStartE2EDuration="3.973009434s" totalImagesPullingTime="1.842363895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:42:44.972691026 +0000 UTC m=+1580.261290616" watchObservedRunningTime="2026-07-23 19:42:44.973009434 +0000 UTC m=+1580.261609026"
Jul 23 19:42:50.966888 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:42:50.966850    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:43:06.858082 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:06.858045    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"]
Jul 23 19:43:06.861760 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:06.861740    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:06.864098 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:06.864080    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:43:06.868871 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:06.868847    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"]
Jul 23 19:43:06.997800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:06.997760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5261b9ad-7a2d-493c-897d-08fff532245b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s\" (UID: \"5261b9ad-7a2d-493c-897d-08fff532245b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:07.099200 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:07.099160    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5261b9ad-7a2d-493c-897d-08fff532245b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s\" (UID: \"5261b9ad-7a2d-493c-897d-08fff532245b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:07.099800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:07.099779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5261b9ad-7a2d-493c-897d-08fff532245b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s\" (UID: \"5261b9ad-7a2d-493c-897d-08fff532245b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:07.173449 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:07.173416    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:07.308032 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:07.308007    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"]
Jul 23 19:43:07.310376 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:43:07.310350    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5261b9ad_7a2d_493c_897d_08fff532245b.slice/crio-ddc7513796777e90208fc11aa40dd6277d7201d8cc1e47b3e108e6bf79b81e74 WatchSource:0}: Error finding container ddc7513796777e90208fc11aa40dd6277d7201d8cc1e47b3e108e6bf79b81e74: Status 404 returned error can't find the container with id ddc7513796777e90208fc11aa40dd6277d7201d8cc1e47b3e108e6bf79b81e74
Jul 23 19:43:08.082226 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:08.082192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s" event={"ID":"5261b9ad-7a2d-493c-897d-08fff532245b","Type":"ContainerStarted","Data":"ddc7513796777e90208fc11aa40dd6277d7201d8cc1e47b3e108e6bf79b81e74"}
Jul 23 19:43:12.103144 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:12.103109    2566 generic.go:356] "Generic (PLEG): container finished" podID="5261b9ad-7a2d-493c-897d-08fff532245b" containerID="765fa43da1c3118ea22c9fa9b0e693ab6e4cfe4cf10d71984ea4ba22ee857a36" exitCode=0
Jul 23 19:43:12.103539 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:12.103197    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s" event={"ID":"5261b9ad-7a2d-493c-897d-08fff532245b","Type":"ContainerDied","Data":"765fa43da1c3118ea22c9fa9b0e693ab6e4cfe4cf10d71984ea4ba22ee857a36"}
Jul 23 19:43:13.250872 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:13.250846    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:13.258357 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:13.258332    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5261b9ad-7a2d-493c-897d-08fff532245b-maas-api-service-ca\") pod \"5261b9ad-7a2d-493c-897d-08fff532245b\" (UID: \"5261b9ad-7a2d-493c-897d-08fff532245b\") "
Jul 23 19:43:13.258713 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:13.258695    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5261b9ad-7a2d-493c-897d-08fff532245b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5261b9ad-7a2d-493c-897d-08fff532245b" (UID: "5261b9ad-7a2d-493c-897d-08fff532245b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:43:13.359207 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:13.359120    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5261b9ad-7a2d-493c-897d-08fff532245b-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:43:14.114098 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:14.114060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s" event={"ID":"5261b9ad-7a2d-493c-897d-08fff532245b","Type":"ContainerDied","Data":"ddc7513796777e90208fc11aa40dd6277d7201d8cc1e47b3e108e6bf79b81e74"}
Jul 23 19:43:14.114262 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:14.114113    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddc7513796777e90208fc11aa40dd6277d7201d8cc1e47b3e108e6bf79b81e74"
Jul 23 19:43:14.114262 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:14.114074    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"
Jul 23 19:43:17.335817 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.335727    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"]
Jul 23 19:43:17.336291 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.336169    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh" podUID="130adbd9-2e38-4017-a401-77ba0906e8ef" containerName="maas-api" containerID="cri-o://28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3" gracePeriod=30
Jul 23 19:43:17.587042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.586977    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:43:17.601201 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.601173    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130adbd9-2e38-4017-a401-77ba0906e8ef-openshift-service-ca\") pod \"130adbd9-2e38-4017-a401-77ba0906e8ef\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") "
Jul 23 19:43:17.601351 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.601213    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h98q8\" (UniqueName: \"kubernetes.io/projected/130adbd9-2e38-4017-a401-77ba0906e8ef-kube-api-access-h98q8\") pod \"130adbd9-2e38-4017-a401-77ba0906e8ef\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") "
Jul 23 19:43:17.601351 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.601262    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls\") pod \"130adbd9-2e38-4017-a401-77ba0906e8ef\" (UID: \"130adbd9-2e38-4017-a401-77ba0906e8ef\") "
Jul 23 19:43:17.601547 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.601522    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/130adbd9-2e38-4017-a401-77ba0906e8ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "130adbd9-2e38-4017-a401-77ba0906e8ef" (UID: "130adbd9-2e38-4017-a401-77ba0906e8ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:43:17.603996 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.603965    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "130adbd9-2e38-4017-a401-77ba0906e8ef" (UID: "130adbd9-2e38-4017-a401-77ba0906e8ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:43:17.604633 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.604605    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/130adbd9-2e38-4017-a401-77ba0906e8ef-kube-api-access-h98q8" (OuterVolumeSpecName: "kube-api-access-h98q8") pod "130adbd9-2e38-4017-a401-77ba0906e8ef" (UID: "130adbd9-2e38-4017-a401-77ba0906e8ef"). InnerVolumeSpecName "kube-api-access-h98q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:43:17.702451 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.702418    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130adbd9-2e38-4017-a401-77ba0906e8ef-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:43:17.702451 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.702445    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130adbd9-2e38-4017-a401-77ba0906e8ef-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:43:17.702451 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:17.702455    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h98q8\" (UniqueName: \"kubernetes.io/projected/130adbd9-2e38-4017-a401-77ba0906e8ef-kube-api-access-h98q8\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:43:18.133736 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.133698    2566 generic.go:356] "Generic (PLEG): container finished" podID="130adbd9-2e38-4017-a401-77ba0906e8ef" containerID="28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3" exitCode=0
Jul 23 19:43:18.133895 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.133751    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"
Jul 23 19:43:18.133895 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.133787    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh" event={"ID":"130adbd9-2e38-4017-a401-77ba0906e8ef","Type":"ContainerDied","Data":"28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3"}
Jul 23 19:43:18.133895 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.133842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh" event={"ID":"130adbd9-2e38-4017-a401-77ba0906e8ef","Type":"ContainerDied","Data":"9b6c941445c126a4c13282dcca0d2d2f7dd61b452a1bcf647973eebf2bc4a2de"}
Jul 23 19:43:18.133895 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.133872    2566 scope.go:117] "RemoveContainer" containerID="28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3"
Jul 23 19:43:18.142764 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.142747    2566 scope.go:117] "RemoveContainer" containerID="28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3"
Jul 23 19:43:18.143034 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:43:18.143012    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3\": container with ID starting with 28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3 not found: ID does not exist" containerID="28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3"
Jul 23 19:43:18.143091 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.143043    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3"} err="failed to get container status \"28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3\": rpc error: code = NotFound desc = could not find container \"28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3\": container with ID starting with 28026e7abc8bc5313724c8764ab148c06eb436185ea521108f77372c435c1bf3 not found: ID does not exist"
Jul 23 19:43:18.153712 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.153682    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"]
Jul 23 19:43:18.159407 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:18.159385    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c543a96f-765df777df-b56fh"]
Jul 23 19:43:19.239290 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:19.239248    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="130adbd9-2e38-4017-a401-77ba0906e8ef" path="/var/lib/kubelet/pods/130adbd9-2e38-4017-a401-77ba0906e8ef/volumes"
Jul 23 19:43:21.945535 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:21.945497    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"]
Jul 23 19:43:21.948219 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:21.948193    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c543a96f-rtf8s"]
Jul 23 19:43:23.243958 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:23.243920    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5261b9ad-7a2d-493c-897d-08fff532245b" path="/var/lib/kubelet/pods/5261b9ad-7a2d-493c-897d-08fff532245b/volumes"
Jul 23 19:43:25.822516 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822484    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"]
Jul 23 19:43:25.822893 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822872    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5261b9ad-7a2d-493c-897d-08fff532245b" containerName="revoke-keys"
Jul 23 19:43:25.822893 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822883    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5261b9ad-7a2d-493c-897d-08fff532245b" containerName="revoke-keys"
Jul 23 19:43:25.822967 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822912    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="130adbd9-2e38-4017-a401-77ba0906e8ef" containerName="maas-api"
Jul 23 19:43:25.822967 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822917    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="130adbd9-2e38-4017-a401-77ba0906e8ef" containerName="maas-api"
Jul 23 19:43:25.823027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822971    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="130adbd9-2e38-4017-a401-77ba0906e8ef" containerName="maas-api"
Jul 23 19:43:25.823027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.822980    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5261b9ad-7a2d-493c-897d-08fff532245b" containerName="revoke-keys"
Jul 23 19:43:25.827172 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.827154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.829651 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.829629    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e21d0ba1-serving-cert\""
Jul 23 19:43:25.829788 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.829721    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:43:25.830407 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.830390    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:43:25.830492 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.830418    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:43:25.836118 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.836094    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"]
Jul 23 19:43:25.876363 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.876336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-maas-api-tls\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.876534 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.876372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmsf\" (UniqueName: \"kubernetes.io/projected/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-kube-api-access-5hmsf\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.876534 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.876415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-openshift-service-ca\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.977100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.977068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-maas-api-tls\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.977284 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.977105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmsf\" (UniqueName: \"kubernetes.io/projected/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-kube-api-access-5hmsf\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.977284 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.977141    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-openshift-service-ca\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.977736 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.977718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-openshift-service-ca\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.979704 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.979659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-maas-api-tls\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:25.986191 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:25.986160    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmsf\" (UniqueName: \"kubernetes.io/projected/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-kube-api-access-5hmsf\") pod \"maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:26.139141 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:26.139048    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:26.276520 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:26.276495    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"]
Jul 23 19:43:26.278788 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:43:26.278761    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7ffdf8_1892_46f6_8254_1826b6fe94fb.slice/crio-46d0d5921814a69b10baec4b18b2b2e624fe1122d704f7d2227499a780d51e8d WatchSource:0}: Error finding container 46d0d5921814a69b10baec4b18b2b2e624fe1122d704f7d2227499a780d51e8d: Status 404 returned error can't find the container with id 46d0d5921814a69b10baec4b18b2b2e624fe1122d704f7d2227499a780d51e8d
Jul 23 19:43:27.179974 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:27.179936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78" event={"ID":"8b7ffdf8-1892-46f6-8254-1826b6fe94fb","Type":"ContainerStarted","Data":"e703d6a24a1f85cf8ce53a9afc102df9d07bd1e0a3ab5538e643c442b0c2c600"}
Jul 23 19:43:27.180371 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:27.179985    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78" event={"ID":"8b7ffdf8-1892-46f6-8254-1826b6fe94fb","Type":"ContainerStarted","Data":"46d0d5921814a69b10baec4b18b2b2e624fe1122d704f7d2227499a780d51e8d"}
Jul 23 19:43:27.180371 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:27.180082    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:27.200937 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:27.200893    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78" podStartSLOduration=1.666940004 podStartE2EDuration="2.200876491s" totalImagesPullingTime="533.936487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:43:27.19903205 +0000 UTC m=+1622.487631643" watchObservedRunningTime="2026-07-23 19:43:27.200876491 +0000 UTC m=+1622.489476082"
Jul 23 19:43:33.190173 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:33.190140    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:43:50.657414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.657378    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"]
Jul 23 19:43:50.665609 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.665576    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:43:50.667197 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.667169    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"]
Jul 23 19:43:50.668052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.668022    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:43:50.719649 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.719616    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c\" (UID: \"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:43:50.820303 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.820262    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c\" (UID: \"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:43:50.820976 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.820947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c\" (UID: \"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:43:50.977596 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:50.977509    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:43:51.112516 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:51.112489    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"]
Jul 23 19:43:51.114682 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:43:51.114640    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78c65c2_974a_4cf5_b5dd_2117e12aeb6f.slice/crio-40d70425c7a7f089095c32bdf9ac49ebb4f1cdf30cf70b20f259170973e3d926 WatchSource:0}: Error finding container 40d70425c7a7f089095c32bdf9ac49ebb4f1cdf30cf70b20f259170973e3d926: Status 404 returned error can't find the container with id 40d70425c7a7f089095c32bdf9ac49ebb4f1cdf30cf70b20f259170973e3d926
Jul 23 19:43:51.294217 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:51.294136    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c" event={"ID":"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f","Type":"ContainerStarted","Data":"40d70425c7a7f089095c32bdf9ac49ebb4f1cdf30cf70b20f259170973e3d926"}
Jul 23 19:43:52.300993 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:52.300960    2566 generic.go:356] "Generic (PLEG): container finished" podID="d78c65c2-974a-4cf5-b5dd-2117e12aeb6f" containerID="da7a49a9cf5e36cbc40377d08a4c4ce5225042ca13eeb89bc4b3a1c34d159810" exitCode=0
Jul 23 19:43:52.301388 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:52.301048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c" event={"ID":"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f","Type":"ContainerDied","Data":"da7a49a9cf5e36cbc40377d08a4c4ce5225042ca13eeb89bc4b3a1c34d159810"}
Jul 23 19:43:53.442510 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:53.442483    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:43:53.545371 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:53.545335    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f-maas-api-service-ca\") pod \"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f\" (UID: \"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f\") "
Jul 23 19:43:53.545770 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:53.545742    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d78c65c2-974a-4cf5-b5dd-2117e12aeb6f" (UID: "d78c65c2-974a-4cf5-b5dd-2117e12aeb6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:43:53.646337 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:53.646257    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:43:54.312443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:54.312405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c" event={"ID":"d78c65c2-974a-4cf5-b5dd-2117e12aeb6f","Type":"ContainerDied","Data":"40d70425c7a7f089095c32bdf9ac49ebb4f1cdf30cf70b20f259170973e3d926"}
Jul 23 19:43:54.312613 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:54.312454    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40d70425c7a7f089095c32bdf9ac49ebb4f1cdf30cf70b20f259170973e3d926"
Jul 23 19:43:54.312613 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:43:54.312420    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"
Jul 23 19:44:01.122074 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.122039    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"]
Jul 23 19:44:01.124611 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.122316    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78" podUID="8b7ffdf8-1892-46f6-8254-1826b6fe94fb" containerName="maas-api" containerID="cri-o://e703d6a24a1f85cf8ce53a9afc102df9d07bd1e0a3ab5538e643c442b0c2c600" gracePeriod=30
Jul 23 19:44:01.353209 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.353179    2566 generic.go:356] "Generic (PLEG): container finished" podID="8b7ffdf8-1892-46f6-8254-1826b6fe94fb" containerID="e703d6a24a1f85cf8ce53a9afc102df9d07bd1e0a3ab5538e643c442b0c2c600" exitCode=0
Jul 23 19:44:01.353369 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.353259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78" event={"ID":"8b7ffdf8-1892-46f6-8254-1826b6fe94fb","Type":"ContainerDied","Data":"e703d6a24a1f85cf8ce53a9afc102df9d07bd1e0a3ab5538e643c442b0c2c600"}
Jul 23 19:44:01.375369 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.375309    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:44:01.422816 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.422781    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-maas-api-tls\") pod \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") "
Jul 23 19:44:01.422970 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.422822    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hmsf\" (UniqueName: \"kubernetes.io/projected/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-kube-api-access-5hmsf\") pod \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") "
Jul 23 19:44:01.422970 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.422863    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-openshift-service-ca\") pod \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\" (UID: \"8b7ffdf8-1892-46f6-8254-1826b6fe94fb\") "
Jul 23 19:44:01.423255 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.423232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b7ffdf8-1892-46f6-8254-1826b6fe94fb" (UID: "8b7ffdf8-1892-46f6-8254-1826b6fe94fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:44:01.425072 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.425046    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b7ffdf8-1892-46f6-8254-1826b6fe94fb" (UID: "8b7ffdf8-1892-46f6-8254-1826b6fe94fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:44:01.425187 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.425164    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-kube-api-access-5hmsf" (OuterVolumeSpecName: "kube-api-access-5hmsf") pod "8b7ffdf8-1892-46f6-8254-1826b6fe94fb" (UID: "8b7ffdf8-1892-46f6-8254-1826b6fe94fb"). InnerVolumeSpecName "kube-api-access-5hmsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:44:01.524753 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.524712    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hmsf\" (UniqueName: \"kubernetes.io/projected/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-kube-api-access-5hmsf\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:01.524753 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.524745    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:01.524753 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:01.524755    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7ffdf8-1892-46f6-8254-1826b6fe94fb-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:02.359866 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:02.359835    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"
Jul 23 19:44:02.360318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:02.359834    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78" event={"ID":"8b7ffdf8-1892-46f6-8254-1826b6fe94fb","Type":"ContainerDied","Data":"46d0d5921814a69b10baec4b18b2b2e624fe1122d704f7d2227499a780d51e8d"}
Jul 23 19:44:02.360318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:02.359967    2566 scope.go:117] "RemoveContainer" containerID="e703d6a24a1f85cf8ce53a9afc102df9d07bd1e0a3ab5538e643c442b0c2c600"
Jul 23 19:44:02.381079 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:02.381054    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"]
Jul 23 19:44:02.385507 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:02.385482    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e21d0ba1-fffff8b7f-mrf78"]
Jul 23 19:44:03.239413 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:03.239383    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7ffdf8-1892-46f6-8254-1826b6fe94fb" path="/var/lib/kubelet/pods/8b7ffdf8-1892-46f6-8254-1826b6fe94fb/volumes"
Jul 23 19:44:05.742211 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:05.742171    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"]
Jul 23 19:44:05.743779 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:05.743757    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e21d0ba1-8lx8c"]
Jul 23 19:44:07.238392 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:07.238356    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d78c65c2-974a-4cf5-b5dd-2117e12aeb6f" path="/var/lib/kubelet/pods/d78c65c2-974a-4cf5-b5dd-2117e12aeb6f/volumes"
Jul 23 19:44:08.931866 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.931834    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"]
Jul 23 19:44:08.932361 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.932339    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78c65c2-974a-4cf5-b5dd-2117e12aeb6f" containerName="revoke-keys"
Jul 23 19:44:08.932418 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.932365    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78c65c2-974a-4cf5-b5dd-2117e12aeb6f" containerName="revoke-keys"
Jul 23 19:44:08.932418 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.932409    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7ffdf8-1892-46f6-8254-1826b6fe94fb" containerName="maas-api"
Jul 23 19:44:08.932418 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.932416    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7ffdf8-1892-46f6-8254-1826b6fe94fb" containerName="maas-api"
Jul 23 19:44:08.932519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.932478    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d78c65c2-974a-4cf5-b5dd-2117e12aeb6f" containerName="revoke-keys"
Jul 23 19:44:08.932519 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.932489    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7ffdf8-1892-46f6-8254-1826b6fe94fb" containerName="maas-api"
Jul 23 19:44:08.937291 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.937271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:08.939844 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.939810    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d3681182-serving-cert\""
Jul 23 19:44:08.939844 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.939832    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:44:08.940012 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.939846    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:44:08.940575 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.940559    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:44:08.946559 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:08.946533    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"]
Jul 23 19:44:09.099660 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.099622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsn84\" (UniqueName: \"kubernetes.io/projected/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-kube-api-access-hsn84\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.099887 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.099787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-openshift-service-ca\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.099887 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.099820    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-maas-api-tls\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.200605 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.200518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsn84\" (UniqueName: \"kubernetes.io/projected/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-kube-api-access-hsn84\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.200605 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.200594    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-openshift-service-ca\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.200865 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.200621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-maas-api-tls\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.201264 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.201241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-openshift-service-ca\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.203315 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.203279    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-maas-api-tls\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.208881 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.208859    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsn84\" (UniqueName: \"kubernetes.io/projected/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-kube-api-access-hsn84\") pod \"maas-api-e2e-ait-d3681182-74486bb89c-zdgxt\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.248861 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.248831    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:09.386417 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.386387    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"]
Jul 23 19:44:09.389500 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:44:09.389467    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded10efeb_cc23_4ce0_9fa4_5e44780489c9.slice/crio-423f9a183927937688fc67f7d943622e306951e6fe6e39a07303ea110fc33ee7 WatchSource:0}: Error finding container 423f9a183927937688fc67f7d943622e306951e6fe6e39a07303ea110fc33ee7: Status 404 returned error can't find the container with id 423f9a183927937688fc67f7d943622e306951e6fe6e39a07303ea110fc33ee7
Jul 23 19:44:09.397049 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:09.397023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt" event={"ID":"ed10efeb-cc23-4ce0-9fa4-5e44780489c9","Type":"ContainerStarted","Data":"423f9a183927937688fc67f7d943622e306951e6fe6e39a07303ea110fc33ee7"}
Jul 23 19:44:10.403641 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:10.403603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt" event={"ID":"ed10efeb-cc23-4ce0-9fa4-5e44780489c9","Type":"ContainerStarted","Data":"e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299"}
Jul 23 19:44:10.404036 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:10.403686    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:10.421286 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:10.421238    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt" podStartSLOduration=1.9920339550000001 podStartE2EDuration="2.421224051s" totalImagesPullingTime="429.190096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:44:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:44:10.41841506 +0000 UTC m=+1665.707014651" watchObservedRunningTime="2026-07-23 19:44:10.421224051 +0000 UTC m=+1665.709823641"
Jul 23 19:44:16.413177 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:16.413148    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:34.589266 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.589231    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"]
Jul 23 19:44:34.594155 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.594140    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:34.596642 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.596618    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:44:34.599497 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.599471    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"]
Jul 23 19:44:34.642139 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.642103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/699f4068-b223-46d5-b2d7-356d696f9be1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3681182-2rxgv\" (UID: \"699f4068-b223-46d5-b2d7-356d696f9be1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:34.743610 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.743578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/699f4068-b223-46d5-b2d7-356d696f9be1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3681182-2rxgv\" (UID: \"699f4068-b223-46d5-b2d7-356d696f9be1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:34.744234 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.744211    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/699f4068-b223-46d5-b2d7-356d696f9be1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3681182-2rxgv\" (UID: \"699f4068-b223-46d5-b2d7-356d696f9be1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:34.905477 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:34.905446    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:35.249240 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:35.249216    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"]
Jul 23 19:44:35.251450 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:44:35.251427    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod699f4068_b223_46d5_b2d7_356d696f9be1.slice/crio-ef65a443d1780eb91843c66687d5232e72b6da1ca6791d375c10d6ba28727b0b WatchSource:0}: Error finding container ef65a443d1780eb91843c66687d5232e72b6da1ca6791d375c10d6ba28727b0b: Status 404 returned error can't find the container with id ef65a443d1780eb91843c66687d5232e72b6da1ca6791d375c10d6ba28727b0b
Jul 23 19:44:35.526518 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:35.526428    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv" event={"ID":"699f4068-b223-46d5-b2d7-356d696f9be1","Type":"ContainerStarted","Data":"c16ecc4a128fcf8f38cff82be79e2f2580133e82d9c41224d0919f705e3a395f"}
Jul 23 19:44:35.526518 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:35.526478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv" event={"ID":"699f4068-b223-46d5-b2d7-356d696f9be1","Type":"ContainerStarted","Data":"ef65a443d1780eb91843c66687d5232e72b6da1ca6791d375c10d6ba28727b0b"}
Jul 23 19:44:35.540664 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:35.540620    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv" podStartSLOduration=1.540599644 podStartE2EDuration="1.540599644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:44:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:44:35.539939735 +0000 UTC m=+1690.828539318" watchObservedRunningTime="2026-07-23 19:44:35.540599644 +0000 UTC m=+1690.829199235"
Jul 23 19:44:36.532919 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:36.532886    2566 generic.go:356] "Generic (PLEG): container finished" podID="699f4068-b223-46d5-b2d7-356d696f9be1" containerID="c16ecc4a128fcf8f38cff82be79e2f2580133e82d9c41224d0919f705e3a395f" exitCode=0
Jul 23 19:44:36.533288 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:36.532964    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv" event={"ID":"699f4068-b223-46d5-b2d7-356d696f9be1","Type":"ContainerDied","Data":"c16ecc4a128fcf8f38cff82be79e2f2580133e82d9c41224d0919f705e3a395f"}
Jul 23 19:44:37.669100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:37.669079    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:37.773331 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:37.773292    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/699f4068-b223-46d5-b2d7-356d696f9be1-maas-api-service-ca\") pod \"699f4068-b223-46d5-b2d7-356d696f9be1\" (UID: \"699f4068-b223-46d5-b2d7-356d696f9be1\") "
Jul 23 19:44:37.773646 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:37.773623    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/699f4068-b223-46d5-b2d7-356d696f9be1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "699f4068-b223-46d5-b2d7-356d696f9be1" (UID: "699f4068-b223-46d5-b2d7-356d696f9be1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:44:37.880763 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:37.875816    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/699f4068-b223-46d5-b2d7-356d696f9be1-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:38.555377 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:38.555347    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"
Jul 23 19:44:38.555542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:38.555342    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv" event={"ID":"699f4068-b223-46d5-b2d7-356d696f9be1","Type":"ContainerDied","Data":"ef65a443d1780eb91843c66687d5232e72b6da1ca6791d375c10d6ba28727b0b"}
Jul 23 19:44:38.555542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:38.555507    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef65a443d1780eb91843c66687d5232e72b6da1ca6791d375c10d6ba28727b0b"
Jul 23 19:44:45.037536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.037415    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"]
Jul 23 19:44:45.038000 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.037707    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt" podUID="ed10efeb-cc23-4ce0-9fa4-5e44780489c9" containerName="maas-api" containerID="cri-o://e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299" gracePeriod=30
Jul 23 19:44:45.288892 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.288841    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:45.346755 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.346727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-openshift-service-ca\") pod \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") "
Jul 23 19:44:45.346928 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.346771    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsn84\" (UniqueName: \"kubernetes.io/projected/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-kube-api-access-hsn84\") pod \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") "
Jul 23 19:44:45.346928 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.346792    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-maas-api-tls\") pod \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\" (UID: \"ed10efeb-cc23-4ce0-9fa4-5e44780489c9\") "
Jul 23 19:44:45.347162 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.347139    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ed10efeb-cc23-4ce0-9fa4-5e44780489c9" (UID: "ed10efeb-cc23-4ce0-9fa4-5e44780489c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:44:45.349016 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.348990    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ed10efeb-cc23-4ce0-9fa4-5e44780489c9" (UID: "ed10efeb-cc23-4ce0-9fa4-5e44780489c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:44:45.349016 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.348995    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-kube-api-access-hsn84" (OuterVolumeSpecName: "kube-api-access-hsn84") pod "ed10efeb-cc23-4ce0-9fa4-5e44780489c9" (UID: "ed10efeb-cc23-4ce0-9fa4-5e44780489c9"). InnerVolumeSpecName "kube-api-access-hsn84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:44:45.448327 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.448243    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsn84\" (UniqueName: \"kubernetes.io/projected/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-kube-api-access-hsn84\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:45.448327 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.448280    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:45.448327 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.448295    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed10efeb-cc23-4ce0-9fa4-5e44780489c9-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:44:45.590781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.590698    2566 generic.go:356] "Generic (PLEG): container finished" podID="ed10efeb-cc23-4ce0-9fa4-5e44780489c9" containerID="e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299" exitCode=0
Jul 23 19:44:45.590781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.590742    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"
Jul 23 19:44:45.590781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.590743    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt" event={"ID":"ed10efeb-cc23-4ce0-9fa4-5e44780489c9","Type":"ContainerDied","Data":"e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299"}
Jul 23 19:44:45.590998 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.590796    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt" event={"ID":"ed10efeb-cc23-4ce0-9fa4-5e44780489c9","Type":"ContainerDied","Data":"423f9a183927937688fc67f7d943622e306951e6fe6e39a07303ea110fc33ee7"}
Jul 23 19:44:45.590998 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.590824    2566 scope.go:117] "RemoveContainer" containerID="e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299"
Jul 23 19:44:45.605884 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.605855    2566 scope.go:117] "RemoveContainer" containerID="e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299"
Jul 23 19:44:45.606173 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:44:45.606151    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299\": container with ID starting with e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299 not found: ID does not exist" containerID="e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299"
Jul 23 19:44:45.606238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.606180    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299"} err="failed to get container status \"e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299\": rpc error: code = NotFound desc = could not find container \"e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299\": container with ID starting with e992685f4fd08fa6a49b62067c3f9089adb562ef93f6a039f7d2cec7ffa33299 not found: ID does not exist"
Jul 23 19:44:45.615206 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.615181    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"]
Jul 23 19:44:45.621241 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:45.621216    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3681182-74486bb89c-zdgxt"]
Jul 23 19:44:47.239614 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:47.239569    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed10efeb-cc23-4ce0-9fa4-5e44780489c9" path="/var/lib/kubelet/pods/ed10efeb-cc23-4ce0-9fa4-5e44780489c9/volumes"
Jul 23 19:44:49.670763 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:49.670724    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"]
Jul 23 19:44:49.673025 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:49.672999    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3681182-2rxgv"]
Jul 23 19:44:51.238283 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:51.238242    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="699f4068-b223-46d5-b2d7-356d696f9be1" path="/var/lib/kubelet/pods/699f4068-b223-46d5-b2d7-356d696f9be1/volumes"
Jul 23 19:44:54.189804 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.189772    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"]
Jul 23 19:44:54.190287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.190271    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="699f4068-b223-46d5-b2d7-356d696f9be1" containerName="revoke-keys"
Jul 23 19:44:54.190326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.190289    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="699f4068-b223-46d5-b2d7-356d696f9be1" containerName="revoke-keys"
Jul 23 19:44:54.190326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.190312    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed10efeb-cc23-4ce0-9fa4-5e44780489c9" containerName="maas-api"
Jul 23 19:44:54.190326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.190321    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed10efeb-cc23-4ce0-9fa4-5e44780489c9" containerName="maas-api"
Jul 23 19:44:54.190416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.190386    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="699f4068-b223-46d5-b2d7-356d696f9be1" containerName="revoke-keys"
Jul 23 19:44:54.190416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.190399    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed10efeb-cc23-4ce0-9fa4-5e44780489c9" containerName="maas-api"
Jul 23 19:44:54.193421 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.193402    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.196414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.196387    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:44:54.196537 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.196416    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:44:54.197027 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.197013    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7db7d650-serving-cert\""
Jul 23 19:44:54.197081 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.197053    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:44:54.204961 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.204940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"]
Jul 23 19:44:54.330601 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.330557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f6a567-521f-41f6-aadd-522dc15669b5-maas-api-tls\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.330761 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.330665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmqmp\" (UniqueName: \"kubernetes.io/projected/62f6a567-521f-41f6-aadd-522dc15669b5-kube-api-access-cmqmp\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.330867 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.330755    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f6a567-521f-41f6-aadd-522dc15669b5-openshift-service-ca\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.432189 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.432157    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f6a567-521f-41f6-aadd-522dc15669b5-maas-api-tls\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.432394 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.432219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmqmp\" (UniqueName: \"kubernetes.io/projected/62f6a567-521f-41f6-aadd-522dc15669b5-kube-api-access-cmqmp\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.432394 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.432243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f6a567-521f-41f6-aadd-522dc15669b5-openshift-service-ca\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.432873 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.432853    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f6a567-521f-41f6-aadd-522dc15669b5-openshift-service-ca\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.434928 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.434903    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f6a567-521f-41f6-aadd-522dc15669b5-maas-api-tls\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.442126 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.442057    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmqmp\" (UniqueName: \"kubernetes.io/projected/62f6a567-521f-41f6-aadd-522dc15669b5-kube-api-access-cmqmp\") pod \"maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.503481 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.503452    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:54.639168 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:54.639142    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"]
Jul 23 19:44:54.641349 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:44:54.641323    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f6a567_521f_41f6_aadd_522dc15669b5.slice/crio-5b77344d1b18e025bf3e9d23528a3a4a4dcbfb78db0eca132125e21e7fbf138a WatchSource:0}: Error finding container 5b77344d1b18e025bf3e9d23528a3a4a4dcbfb78db0eca132125e21e7fbf138a: Status 404 returned error can't find the container with id 5b77344d1b18e025bf3e9d23528a3a4a4dcbfb78db0eca132125e21e7fbf138a
Jul 23 19:44:55.642193 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:55.642151    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn" event={"ID":"62f6a567-521f-41f6-aadd-522dc15669b5","Type":"ContainerStarted","Data":"37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468"}
Jul 23 19:44:55.642569 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:55.642218    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:44:55.642569 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:55.642235    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn" event={"ID":"62f6a567-521f-41f6-aadd-522dc15669b5","Type":"ContainerStarted","Data":"5b77344d1b18e025bf3e9d23528a3a4a4dcbfb78db0eca132125e21e7fbf138a"}
Jul 23 19:44:55.660177 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:44:55.660126    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn" podStartSLOduration=1.242597991 podStartE2EDuration="1.660111409s" totalImagesPullingTime="417.513418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:44:55.658258916 +0000 UTC m=+1710.946858507" watchObservedRunningTime="2026-07-23 19:44:55.660111409 +0000 UTC m=+1710.948711000"
Jul 23 19:45:00.132451 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.132416    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"]
Jul 23 19:45:00.136278 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.136238    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"]
Jul 23 19:45:00.136418 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.136345    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"
Jul 23 19:45:00.138686 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.138651    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:45:00.139538 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.139522    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"
Jul 23 19:45:00.142834 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.142804    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"]
Jul 23 19:45:00.146056 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.146035    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"]
Jul 23 19:45:00.151528 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.151506    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"
Jul 23 19:45:00.158581 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.158558    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"
Jul 23 19:45:00.305850 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.305819    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"]
Jul 23 19:45:00.306957 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:45:00.306906    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f336c71_759e_45a1_94ae_cf8d63a69573.slice/crio-910a22fcd21df1ffbaa93cadaa48f17f5636d7dd792279d0a2a2a9ff2e250c42 WatchSource:0}: Error finding container 910a22fcd21df1ffbaa93cadaa48f17f5636d7dd792279d0a2a2a9ff2e250c42: Status 404 returned error can't find the container with id 910a22fcd21df1ffbaa93cadaa48f17f5636d7dd792279d0a2a2a9ff2e250c42
Jul 23 19:45:00.332287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.332263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"]
Jul 23 19:45:00.333859 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:45:00.333836    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cffb93f_df12_4af0_957c_a7bc1f97b130.slice/crio-6a2338de4076ed2f5edda84ae3c68df996a42e503f6f9e8194ef7d2d33134ced WatchSource:0}: Error finding container 6a2338de4076ed2f5edda84ae3c68df996a42e503f6f9e8194ef7d2d33134ced: Status 404 returned error can't find the container with id 6a2338de4076ed2f5edda84ae3c68df996a42e503f6f9e8194ef7d2d33134ced
Jul 23 19:45:00.667848 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.667811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw" event={"ID":"7f336c71-759e-45a1-94ae-cf8d63a69573","Type":"ContainerStarted","Data":"910a22fcd21df1ffbaa93cadaa48f17f5636d7dd792279d0a2a2a9ff2e250c42"}
Jul 23 19:45:00.669017 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:00.668992    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn" event={"ID":"8cffb93f-df12-4af0-957c-a7bc1f97b130","Type":"ContainerStarted","Data":"6a2338de4076ed2f5edda84ae3c68df996a42e503f6f9e8194ef7d2d33134ced"}
Jul 23 19:45:01.653814 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:01.653784    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:45:01.676254 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:01.676213    2566 generic.go:356] "Generic (PLEG): container finished" podID="7f336c71-759e-45a1-94ae-cf8d63a69573" containerID="7777d391cf11541b7316b420f9e33b901ca321dc6dfeef8cfb0e3f31a2421308" exitCode=0
Jul 23 19:45:01.676422 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:01.676302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw" event={"ID":"7f336c71-759e-45a1-94ae-cf8d63a69573","Type":"ContainerDied","Data":"7777d391cf11541b7316b420f9e33b901ca321dc6dfeef8cfb0e3f31a2421308"}
Jul 23 19:45:01.677824 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:01.677797    2566 generic.go:356] "Generic (PLEG): container finished" podID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerID="25b7b3ba7076dcd8192857ec99dc655681e1649b89a25aa16d69b549b0c9c579" exitCode=7
Jul 23 19:45:01.677944 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:01.677842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn" event={"ID":"8cffb93f-df12-4af0-957c-a7bc1f97b130","Type":"ContainerDied","Data":"25b7b3ba7076dcd8192857ec99dc655681e1649b89a25aa16d69b549b0c9c579"}
Jul 23 19:45:01.678015 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:01.678004    2566 scope.go:117] "RemoveContainer" containerID="25b7b3ba7076dcd8192857ec99dc655681e1649b89a25aa16d69b549b0c9c579"
Jul 23 19:45:02.684587 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:02.684544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn" event={"ID":"8cffb93f-df12-4af0-957c-a7bc1f97b130","Type":"ContainerStarted","Data":"085484c1ab7f1ec0e7dc8cce9d0e4bd34ac13b997235daf088e4a29726299831"}
Jul 23 19:45:02.699351 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:02.699293    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn" podStartSLOduration=2.699273004 podStartE2EDuration="2.699273004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:45:02.697980374 +0000 UTC m=+1717.986579965" watchObservedRunningTime="2026-07-23 19:45:02.699273004 +0000 UTC m=+1717.987872602"
Jul 23 19:45:02.810069 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:02.810042    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"
Jul 23 19:45:03.692052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:03.692024    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw"
Jul 23 19:45:03.692052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:03.692028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747265-95gkw" event={"ID":"7f336c71-759e-45a1-94ae-cf8d63a69573","Type":"ContainerDied","Data":"910a22fcd21df1ffbaa93cadaa48f17f5636d7dd792279d0a2a2a9ff2e250c42"}
Jul 23 19:45:03.692550 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:03.692084    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="910a22fcd21df1ffbaa93cadaa48f17f5636d7dd792279d0a2a2a9ff2e250c42"
Jul 23 19:45:03.695748 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:03.695714    2566 generic.go:356] "Generic (PLEG): container finished" podID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerID="085484c1ab7f1ec0e7dc8cce9d0e4bd34ac13b997235daf088e4a29726299831" exitCode=0
Jul 23 19:45:03.695868 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:03.695758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn" event={"ID":"8cffb93f-df12-4af0-957c-a7bc1f97b130","Type":"ContainerDied","Data":"085484c1ab7f1ec0e7dc8cce9d0e4bd34ac13b997235daf088e4a29726299831"}
Jul 23 19:45:03.695868 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:03.695816    2566 scope.go:117] "RemoveContainer" containerID="25b7b3ba7076dcd8192857ec99dc655681e1649b89a25aa16d69b549b0c9c579"
Jul 23 19:45:04.838592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:04.838565    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"
Jul 23 19:45:05.708880 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:05.708848    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"
Jul 23 19:45:05.708880 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:05.708854    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn" event={"ID":"8cffb93f-df12-4af0-957c-a7bc1f97b130","Type":"ContainerDied","Data":"6a2338de4076ed2f5edda84ae3c68df996a42e503f6f9e8194ef7d2d33134ced"}
Jul 23 19:45:05.709083 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:05.708905    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a2338de4076ed2f5edda84ae3c68df996a42e503f6f9e8194ef7d2d33134ced"
Jul 23 19:45:18.931780 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.931743    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"]
Jul 23 19:45:18.932223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932127    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f336c71-759e-45a1-94ae-cf8d63a69573" containerName="cleanup"
Jul 23 19:45:18.932223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932137    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f336c71-759e-45a1-94ae-cf8d63a69573" containerName="cleanup"
Jul 23 19:45:18.932223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932150    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerName="cleanup"
Jul 23 19:45:18.932223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932156    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerName="cleanup"
Jul 23 19:45:18.932223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932165    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerName="cleanup"
Jul 23 19:45:18.932223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932170    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerName="cleanup"
Jul 23 19:45:18.932415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932233    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerName="cleanup"
Jul 23 19:45:18.932415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932242    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" containerName="cleanup"
Jul 23 19:45:18.932415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.932248    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f336c71-759e-45a1-94ae-cf8d63a69573" containerName="cleanup"
Jul 23 19:45:18.935569 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.935553    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:18.937799 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.937774    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:45:18.942356 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:18.942328    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"]
Jul 23 19:45:19.048501 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.048458    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n\" (UID: \"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:19.149418 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.149387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n\" (UID: \"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:19.150022 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.150003    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n\" (UID: \"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:19.249420 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.249347    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:19.385170 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.385150    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"]
Jul 23 19:45:19.387774 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:45:19.387748    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8d0cf2_0fd4_4d84_b2d0_2417bb1827a1.slice/crio-281b00002ae7760334e4848c12c3e044562bafafb60d06550d7c6dd996bbad16 WatchSource:0}: Error finding container 281b00002ae7760334e4848c12c3e044562bafafb60d06550d7c6dd996bbad16: Status 404 returned error can't find the container with id 281b00002ae7760334e4848c12c3e044562bafafb60d06550d7c6dd996bbad16
Jul 23 19:45:19.788488 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.788457    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n" event={"ID":"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1","Type":"ContainerStarted","Data":"1675efca7d80c1550b9695cf2bcc99f9c747237d575d170d43321a50ce63ce90"}
Jul 23 19:45:19.788696 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.788514    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n" event={"ID":"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1","Type":"ContainerStarted","Data":"281b00002ae7760334e4848c12c3e044562bafafb60d06550d7c6dd996bbad16"}
Jul 23 19:45:19.803454 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:19.803406    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n" podStartSLOduration=1.803391615 podStartE2EDuration="1.803391615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:45:19.801927509 +0000 UTC m=+1735.090527102" watchObservedRunningTime="2026-07-23 19:45:19.803391615 +0000 UTC m=+1735.091991215"
Jul 23 19:45:20.794752 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:20.794713    2566 generic.go:356] "Generic (PLEG): container finished" podID="5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1" containerID="1675efca7d80c1550b9695cf2bcc99f9c747237d575d170d43321a50ce63ce90" exitCode=0
Jul 23 19:45:20.795104 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:20.794785    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n" event={"ID":"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1","Type":"ContainerDied","Data":"1675efca7d80c1550b9695cf2bcc99f9c747237d575d170d43321a50ce63ce90"}
Jul 23 19:45:21.925238 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:21.925216    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:21.975035 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:21.975006    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1-maas-api-service-ca\") pod \"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1\" (UID: \"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1\") "
Jul 23 19:45:21.975336 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:21.975313    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1" (UID: "5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:45:22.076410 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:22.076336    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:45:22.806358 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:22.806329    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"
Jul 23 19:45:22.806528 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:22.806364    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n" event={"ID":"5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1","Type":"ContainerDied","Data":"281b00002ae7760334e4848c12c3e044562bafafb60d06550d7c6dd996bbad16"}
Jul 23 19:45:22.806528 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:22.806413    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="281b00002ae7760334e4848c12c3e044562bafafb60d06550d7c6dd996bbad16"
Jul 23 19:45:29.370147 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.370109    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"]
Jul 23 19:45:29.370702 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.370357    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn" podUID="62f6a567-521f-41f6-aadd-522dc15669b5" containerName="maas-api" containerID="cri-o://37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468" gracePeriod=30
Jul 23 19:45:29.394932 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.394907    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"]
Jul 23 19:45:29.398552 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.398531    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7db7d650-29747265-c7jrn"]
Jul 23 19:45:29.620654 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.620597    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:45:29.746232 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.746196    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmqmp\" (UniqueName: \"kubernetes.io/projected/62f6a567-521f-41f6-aadd-522dc15669b5-kube-api-access-cmqmp\") pod \"62f6a567-521f-41f6-aadd-522dc15669b5\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") "
Jul 23 19:45:29.746415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.746280    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f6a567-521f-41f6-aadd-522dc15669b5-maas-api-tls\") pod \"62f6a567-521f-41f6-aadd-522dc15669b5\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") "
Jul 23 19:45:29.746415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.746392    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f6a567-521f-41f6-aadd-522dc15669b5-openshift-service-ca\") pod \"62f6a567-521f-41f6-aadd-522dc15669b5\" (UID: \"62f6a567-521f-41f6-aadd-522dc15669b5\") "
Jul 23 19:45:29.746751 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.746725    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f6a567-521f-41f6-aadd-522dc15669b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62f6a567-521f-41f6-aadd-522dc15669b5" (UID: "62f6a567-521f-41f6-aadd-522dc15669b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:45:29.748475 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.748453    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f6a567-521f-41f6-aadd-522dc15669b5-kube-api-access-cmqmp" (OuterVolumeSpecName: "kube-api-access-cmqmp") pod "62f6a567-521f-41f6-aadd-522dc15669b5" (UID: "62f6a567-521f-41f6-aadd-522dc15669b5"). InnerVolumeSpecName "kube-api-access-cmqmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:45:29.748568 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.748548    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62f6a567-521f-41f6-aadd-522dc15669b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62f6a567-521f-41f6-aadd-522dc15669b5" (UID: "62f6a567-521f-41f6-aadd-522dc15669b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:45:29.842562 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.842527    2566 generic.go:356] "Generic (PLEG): container finished" podID="62f6a567-521f-41f6-aadd-522dc15669b5" containerID="37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468" exitCode=0
Jul 23 19:45:29.842734 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.842571    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"
Jul 23 19:45:29.842734 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.842585    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn" event={"ID":"62f6a567-521f-41f6-aadd-522dc15669b5","Type":"ContainerDied","Data":"37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468"}
Jul 23 19:45:29.842734 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.842638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn" event={"ID":"62f6a567-521f-41f6-aadd-522dc15669b5","Type":"ContainerDied","Data":"5b77344d1b18e025bf3e9d23528a3a4a4dcbfb78db0eca132125e21e7fbf138a"}
Jul 23 19:45:29.842734 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.842660    2566 scope.go:117] "RemoveContainer" containerID="37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468"
Jul 23 19:45:29.848184 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.848159    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f6a567-521f-41f6-aadd-522dc15669b5-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:45:29.848292 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.848189    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmqmp\" (UniqueName: \"kubernetes.io/projected/62f6a567-521f-41f6-aadd-522dc15669b5-kube-api-access-cmqmp\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:45:29.848292 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.848205    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f6a567-521f-41f6-aadd-522dc15669b5-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:45:29.851779 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.851763    2566 scope.go:117] "RemoveContainer" containerID="37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468"
Jul 23 19:45:29.852050 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:45:29.852031    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468\": container with ID starting with 37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468 not found: ID does not exist" containerID="37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468"
Jul 23 19:45:29.852093 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.852060    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468"} err="failed to get container status \"37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468\": rpc error: code = NotFound desc = could not find container \"37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468\": container with ID starting with 37b4b34510f4a11875f2885e9f75eaaa697c19a213962f430cd58bde913d8468 not found: ID does not exist"
Jul 23 19:45:29.864163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.864143    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"]
Jul 23 19:45:29.867637 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:29.867617    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7db7d650-5744d54dbd-58mdn"]
Jul 23 19:45:31.245270 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:31.245227    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f6a567-521f-41f6-aadd-522dc15669b5" path="/var/lib/kubelet/pods/62f6a567-521f-41f6-aadd-522dc15669b5/volumes"
Jul 23 19:45:31.245632 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:31.245556    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cffb93f-df12-4af0-957c-a7bc1f97b130" path="/var/lib/kubelet/pods/8cffb93f-df12-4af0-957c-a7bc1f97b130/volumes"
Jul 23 19:45:34.014150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:34.014117    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"]
Jul 23 19:45:34.016779 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:34.016754    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7db7d650-2pk5n"]
Jul 23 19:45:35.238937 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:45:35.238905    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1" path="/var/lib/kubelet/pods/5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1/volumes"
Jul 23 19:47:13.709127 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"]
Jul 23 19:47:13.709728 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709571    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f6a567-521f-41f6-aadd-522dc15669b5" containerName="maas-api"
Jul 23 19:47:13.709728 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709583    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f6a567-521f-41f6-aadd-522dc15669b5" containerName="maas-api"
Jul 23 19:47:13.709728 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709592    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1" containerName="revoke-keys"
Jul 23 19:47:13.709728 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709598    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1" containerName="revoke-keys"
Jul 23 19:47:13.709728 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709694    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f6a567-521f-41f6-aadd-522dc15669b5" containerName="maas-api"
Jul 23 19:47:13.709728 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.709711    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8d0cf2-0fd4-4d84-b2d0-2417bb1827a1" containerName="revoke-keys"
Jul 23 19:47:13.712443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.712423    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.715106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.715084    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3c342296-serving-cert\""
Jul 23 19:47:13.715276 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.715256    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:47:13.715353 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.715316    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:47:13.715995 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.715977    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:47:13.721500 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.721477    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"]
Jul 23 19:47:13.734844 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.734823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.734944 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.734874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d91d30b-cf12-40bb-9168-f1fb045e6231-openshift-service-ca\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.734994 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.734956    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwcwm\" (UniqueName: \"kubernetes.io/projected/9d91d30b-cf12-40bb-9168-f1fb045e6231-kube-api-access-lwcwm\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.836248 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.836207    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwcwm\" (UniqueName: \"kubernetes.io/projected/9d91d30b-cf12-40bb-9168-f1fb045e6231-kube-api-access-lwcwm\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.836424 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.836266    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.836424 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.836316    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d91d30b-cf12-40bb-9168-f1fb045e6231-openshift-service-ca\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.836496 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:47:13.836420    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-serving-cert: secret "maas-api-e2e-mt-3c342296-serving-cert" not found
Jul 23 19:47:13.836496 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:47:13.836484    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls podName:9d91d30b-cf12-40bb-9168-f1fb045e6231 nodeName:}" failed. No retries permitted until 2026-07-23 19:47:14.336468307 +0000 UTC m=+1849.625067876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls") pod "maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" (UID: "9d91d30b-cf12-40bb-9168-f1fb045e6231") : secret "maas-api-e2e-mt-3c342296-serving-cert" not found
Jul 23 19:47:13.836924 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.836908    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d91d30b-cf12-40bb-9168-f1fb045e6231-openshift-service-ca\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:13.844617 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:13.844586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwcwm\" (UniqueName: \"kubernetes.io/projected/9d91d30b-cf12-40bb-9168-f1fb045e6231-kube-api-access-lwcwm\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:14.341014 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:14.340976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:14.343471 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:14.343442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls\") pod \"maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:14.625619 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:14.625544    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:14.765408 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:14.765380    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"]
Jul 23 19:47:14.767466 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:47:14.767428    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d91d30b_cf12_40bb_9168_f1fb045e6231.slice/crio-11d20765c3f47a8efee1b8f59027e85afe50371bf42d38422cf580572b25ff07 WatchSource:0}: Error finding container 11d20765c3f47a8efee1b8f59027e85afe50371bf42d38422cf580572b25ff07: Status 404 returned error can't find the container with id 11d20765c3f47a8efee1b8f59027e85afe50371bf42d38422cf580572b25ff07
Jul 23 19:47:15.367047 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:15.367015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" event={"ID":"9d91d30b-cf12-40bb-9168-f1fb045e6231","Type":"ContainerStarted","Data":"145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4"}
Jul 23 19:47:15.367215 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:15.367064    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" event={"ID":"9d91d30b-cf12-40bb-9168-f1fb045e6231","Type":"ContainerStarted","Data":"11d20765c3f47a8efee1b8f59027e85afe50371bf42d38422cf580572b25ff07"}
Jul 23 19:47:15.367215 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:15.367113    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:15.385436 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:15.385357    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" podStartSLOduration=2.024614857 podStartE2EDuration="2.38534407s" totalImagesPullingTime="360.729213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:47:15.383603501 +0000 UTC m=+1850.672203092" watchObservedRunningTime="2026-07-23 19:47:15.38534407 +0000 UTC m=+1850.673943662"
Jul 23 19:47:21.377817 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:21.377789    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:24.955701 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:24.955649    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"]
Jul 23 19:47:24.965829 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:24.965796    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:24.967222 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:24.967191    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"]
Jul 23 19:47:24.968850 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:24.968827    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:47:25.040732 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:25.040698    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d214b6-6837-4ce9-bbfb-ec01d7543d38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3c342296-9wcdj\" (UID: \"c1d214b6-6837-4ce9-bbfb-ec01d7543d38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:25.142152 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:25.142114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d214b6-6837-4ce9-bbfb-ec01d7543d38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3c342296-9wcdj\" (UID: \"c1d214b6-6837-4ce9-bbfb-ec01d7543d38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:25.142800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:25.142782    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d214b6-6837-4ce9-bbfb-ec01d7543d38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3c342296-9wcdj\" (UID: \"c1d214b6-6837-4ce9-bbfb-ec01d7543d38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:25.281570 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:25.281498    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:47:25.290354 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:25.290336    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:25.425529 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:25.425504    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"]
Jul 23 19:47:25.427074 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:47:25.427050    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d214b6_6837_4ce9_bbfb_ec01d7543d38.slice/crio-5b822747a4b42174e7aebdf2bc408391af88b693a6d40fc0e3d4417d77de19d3 WatchSource:0}: Error finding container 5b822747a4b42174e7aebdf2bc408391af88b693a6d40fc0e3d4417d77de19d3: Status 404 returned error can't find the container with id 5b822747a4b42174e7aebdf2bc408391af88b693a6d40fc0e3d4417d77de19d3
Jul 23 19:47:26.425746 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:26.425717    2566 generic.go:356] "Generic (PLEG): container finished" podID="c1d214b6-6837-4ce9-bbfb-ec01d7543d38" containerID="25b48e51420a3d9ff894a70a56a34df1948a0ac7fa59ee5021fa78e53b2b0267" exitCode=0
Jul 23 19:47:26.426130 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:26.425808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj" event={"ID":"c1d214b6-6837-4ce9-bbfb-ec01d7543d38","Type":"ContainerDied","Data":"25b48e51420a3d9ff894a70a56a34df1948a0ac7fa59ee5021fa78e53b2b0267"}
Jul 23 19:47:26.426130 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:26.425856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj" event={"ID":"c1d214b6-6837-4ce9-bbfb-ec01d7543d38","Type":"ContainerStarted","Data":"5b822747a4b42174e7aebdf2bc408391af88b693a6d40fc0e3d4417d77de19d3"}
Jul 23 19:47:27.561523 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:27.561492    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:27.667764 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:27.667730    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d214b6-6837-4ce9-bbfb-ec01d7543d38-maas-api-service-ca\") pod \"c1d214b6-6837-4ce9-bbfb-ec01d7543d38\" (UID: \"c1d214b6-6837-4ce9-bbfb-ec01d7543d38\") "
Jul 23 19:47:27.668045 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:27.668027    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1d214b6-6837-4ce9-bbfb-ec01d7543d38-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1d214b6-6837-4ce9-bbfb-ec01d7543d38" (UID: "c1d214b6-6837-4ce9-bbfb-ec01d7543d38"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:47:27.769201 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:27.769118    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d214b6-6837-4ce9-bbfb-ec01d7543d38-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:47:28.437264 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:28.437233    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"
Jul 23 19:47:28.437264 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:28.437254    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj" event={"ID":"c1d214b6-6837-4ce9-bbfb-ec01d7543d38","Type":"ContainerDied","Data":"5b822747a4b42174e7aebdf2bc408391af88b693a6d40fc0e3d4417d77de19d3"}
Jul 23 19:47:28.437951 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:28.437319    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b822747a4b42174e7aebdf2bc408391af88b693a6d40fc0e3d4417d77de19d3"
Jul 23 19:47:35.473532 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.473494    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"]
Jul 23 19:47:35.473922 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.473741    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" podUID="9d91d30b-cf12-40bb-9168-f1fb045e6231" containerName="maas-api" containerID="cri-o://145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4" gracePeriod=30
Jul 23 19:47:35.725764 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.725702    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:35.845374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.845343    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwcwm\" (UniqueName: \"kubernetes.io/projected/9d91d30b-cf12-40bb-9168-f1fb045e6231-kube-api-access-lwcwm\") pod \"9d91d30b-cf12-40bb-9168-f1fb045e6231\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") "
Jul 23 19:47:35.845374 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.845393    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d91d30b-cf12-40bb-9168-f1fb045e6231-openshift-service-ca\") pod \"9d91d30b-cf12-40bb-9168-f1fb045e6231\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") "
Jul 23 19:47:35.845592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.845419    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls\") pod \"9d91d30b-cf12-40bb-9168-f1fb045e6231\" (UID: \"9d91d30b-cf12-40bb-9168-f1fb045e6231\") "
Jul 23 19:47:35.845768 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.845744    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d91d30b-cf12-40bb-9168-f1fb045e6231-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9d91d30b-cf12-40bb-9168-f1fb045e6231" (UID: "9d91d30b-cf12-40bb-9168-f1fb045e6231"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:47:35.847738 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.847712    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9d91d30b-cf12-40bb-9168-f1fb045e6231" (UID: "9d91d30b-cf12-40bb-9168-f1fb045e6231"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:47:35.847833 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.847757    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d91d30b-cf12-40bb-9168-f1fb045e6231-kube-api-access-lwcwm" (OuterVolumeSpecName: "kube-api-access-lwcwm") pod "9d91d30b-cf12-40bb-9168-f1fb045e6231" (UID: "9d91d30b-cf12-40bb-9168-f1fb045e6231"). InnerVolumeSpecName "kube-api-access-lwcwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:47:35.947015 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.946969    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d91d30b-cf12-40bb-9168-f1fb045e6231-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:47:35.947015 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.947010    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d91d30b-cf12-40bb-9168-f1fb045e6231-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:47:35.947015 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:35.947019    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwcwm\" (UniqueName: \"kubernetes.io/projected/9d91d30b-cf12-40bb-9168-f1fb045e6231-kube-api-access-lwcwm\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:47:36.482042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.482009    2566 generic.go:356] "Generic (PLEG): container finished" podID="9d91d30b-cf12-40bb-9168-f1fb045e6231" containerID="145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4" exitCode=0
Jul 23 19:47:36.482443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.482058    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"
Jul 23 19:47:36.482443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.482092    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" event={"ID":"9d91d30b-cf12-40bb-9168-f1fb045e6231","Type":"ContainerDied","Data":"145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4"}
Jul 23 19:47:36.482443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.482147    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh" event={"ID":"9d91d30b-cf12-40bb-9168-f1fb045e6231","Type":"ContainerDied","Data":"11d20765c3f47a8efee1b8f59027e85afe50371bf42d38422cf580572b25ff07"}
Jul 23 19:47:36.482443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.482175    2566 scope.go:117] "RemoveContainer" containerID="145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4"
Jul 23 19:47:36.491883 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.491865    2566 scope.go:117] "RemoveContainer" containerID="145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4"
Jul 23 19:47:36.492131 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:47:36.492113    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4\": container with ID starting with 145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4 not found: ID does not exist" containerID="145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4"
Jul 23 19:47:36.492169 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.492140    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4"} err="failed to get container status \"145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4\": rpc error: code = NotFound desc = could not find container \"145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4\": container with ID starting with 145074bc3de4c02dc8bd0df5efca5d10d014e92d30de56d876a0041ce6716da4 not found: ID does not exist"
Jul 23 19:47:36.504563 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.504540    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"]
Jul 23 19:47:36.510970 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:36.510950    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c342296-7d66b56b6c-b9xrh"]
Jul 23 19:47:37.238875 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:37.238840    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d91d30b-cf12-40bb-9168-f1fb045e6231" path="/var/lib/kubelet/pods/9d91d30b-cf12-40bb-9168-f1fb045e6231/volumes"
Jul 23 19:47:40.051195 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:40.051146    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"]
Jul 23 19:47:40.052997 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:40.052974    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3c342296-9wcdj"]
Jul 23 19:47:41.239162 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:47:41.239128    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d214b6-6837-4ce9-bbfb-ec01d7543d38" path="/var/lib/kubelet/pods/c1d214b6-6837-4ce9-bbfb-ec01d7543d38/volumes"
Jul 23 19:48:08.820658 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.820580    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"]
Jul 23 19:48:08.821122 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.820994    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d91d30b-cf12-40bb-9168-f1fb045e6231" containerName="maas-api"
Jul 23 19:48:08.821122 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.821007    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d91d30b-cf12-40bb-9168-f1fb045e6231" containerName="maas-api"
Jul 23 19:48:08.821122 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.821023    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d214b6-6837-4ce9-bbfb-ec01d7543d38" containerName="revoke-keys"
Jul 23 19:48:08.821122 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.821029    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d214b6-6837-4ce9-bbfb-ec01d7543d38" containerName="revoke-keys"
Jul 23 19:48:08.821122 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.821092    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d91d30b-cf12-40bb-9168-f1fb045e6231" containerName="maas-api"
Jul 23 19:48:08.821122 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.821102    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d214b6-6837-4ce9-bbfb-ec01d7543d38" containerName="revoke-keys"
Jul 23 19:48:08.824326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.824310    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:08.826715 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.826690    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f275417d-serving-cert\""
Jul 23 19:48:08.826715 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.826710    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:48:08.826911 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.826730    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:48:08.827493 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.827451    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:48:08.833287 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.833263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"]
Jul 23 19:48:08.948583 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.948540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:08.948821 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.948593    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-openshift-service-ca\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:08.948821 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:08.948740    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l86qg\" (UniqueName: \"kubernetes.io/projected/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-kube-api-access-l86qg\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.050162 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.050124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l86qg\" (UniqueName: \"kubernetes.io/projected/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-kube-api-access-l86qg\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.050370 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.050213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.050370 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.050255    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-openshift-service-ca\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.050370 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:48:09.050352    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-serving-cert: secret "maas-api-e2e-mt-f275417d-serving-cert" not found
Jul 23 19:48:09.050546 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:48:09.050432    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls podName:f8a79cfb-c0d0-4429-abdb-b0994b920f6f nodeName:}" failed. No retries permitted until 2026-07-23 19:48:09.550414881 +0000 UTC m=+1904.839014450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls") pod "maas-api-e2e-mt-f275417d-797b59845-n8jjh" (UID: "f8a79cfb-c0d0-4429-abdb-b0994b920f6f") : secret "maas-api-e2e-mt-f275417d-serving-cert" not found
Jul 23 19:48:09.050978 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.050956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-openshift-service-ca\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.061334 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.061305    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l86qg\" (UniqueName: \"kubernetes.io/projected/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-kube-api-access-l86qg\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.555098 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.555059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.557593 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.557572    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls\") pod \"maas-api-e2e-mt-f275417d-797b59845-n8jjh\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.736802 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.736767    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:09.869595 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.869569    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"]
Jul 23 19:48:09.871435 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:48:09.871407    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a79cfb_c0d0_4429_abdb_b0994b920f6f.slice/crio-b3279c564269cced6d863e53d9b071995c789fcb9d692941e66f4dcc4083b460 WatchSource:0}: Error finding container b3279c564269cced6d863e53d9b071995c789fcb9d692941e66f4dcc4083b460: Status 404 returned error can't find the container with id b3279c564269cced6d863e53d9b071995c789fcb9d692941e66f4dcc4083b460
Jul 23 19:48:09.872579 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:09.872522    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:48:10.662594 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:10.662559    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh" event={"ID":"f8a79cfb-c0d0-4429-abdb-b0994b920f6f","Type":"ContainerStarted","Data":"3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a"}
Jul 23 19:48:10.662817 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:10.662614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh" event={"ID":"f8a79cfb-c0d0-4429-abdb-b0994b920f6f","Type":"ContainerStarted","Data":"b3279c564269cced6d863e53d9b071995c789fcb9d692941e66f4dcc4083b460"}
Jul 23 19:48:10.662817 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:10.662708    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:10.680067 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:10.680018    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh" podStartSLOduration=2.388780526 podStartE2EDuration="2.680003163s" totalImagesPullingTime="291.222637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:48:10.677964796 +0000 UTC m=+1905.966564387" watchObservedRunningTime="2026-07-23 19:48:10.680003163 +0000 UTC m=+1905.968602753"
Jul 23 19:48:16.674312 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:16.674279    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:48:51.561910 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.561874    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"]
Jul 23 19:48:51.565454 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.565439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:51.568415 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.568397    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:48:51.572326 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.572285    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"]
Jul 23 19:48:51.652433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.652402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f275417d-4rhkm\" (UID: \"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:51.753400 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.753365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f275417d-4rhkm\" (UID: \"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:51.754017 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.753996    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f275417d-4rhkm\" (UID: \"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:51.876196 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:51.876112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:52.216336 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:52.216309    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"]
Jul 23 19:48:52.217873 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:48:52.217836    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea6249d_4ed9_4504_8cc7_fa0e2af8870f.slice/crio-bcd51ea52af983cf82bf0258b12bb40acc5fc4f27dea9582f70c3de8eb274a9f WatchSource:0}: Error finding container bcd51ea52af983cf82bf0258b12bb40acc5fc4f27dea9582f70c3de8eb274a9f: Status 404 returned error can't find the container with id bcd51ea52af983cf82bf0258b12bb40acc5fc4f27dea9582f70c3de8eb274a9f
Jul 23 19:48:52.888396 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:52.888299    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm" event={"ID":"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f","Type":"ContainerStarted","Data":"e90aac6a501db7ca46acac953b92bde6511e34d556145438e74f8c874073a948"}
Jul 23 19:48:52.888396 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:52.888360    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm" event={"ID":"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f","Type":"ContainerStarted","Data":"bcd51ea52af983cf82bf0258b12bb40acc5fc4f27dea9582f70c3de8eb274a9f"}
Jul 23 19:48:52.903840 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:52.903794    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm" podStartSLOduration=1.9037794369999999 podStartE2EDuration="1.903779437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:48:52.902929746 +0000 UTC m=+1948.191529337" watchObservedRunningTime="2026-07-23 19:48:52.903779437 +0000 UTC m=+1948.192379028"
Jul 23 19:48:53.895631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:53.895595    2566 generic.go:356] "Generic (PLEG): container finished" podID="6ea6249d-4ed9-4504-8cc7-fa0e2af8870f" containerID="e90aac6a501db7ca46acac953b92bde6511e34d556145438e74f8c874073a948" exitCode=0
Jul 23 19:48:53.896114 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:53.895640    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm" event={"ID":"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f","Type":"ContainerDied","Data":"e90aac6a501db7ca46acac953b92bde6511e34d556145438e74f8c874073a948"}
Jul 23 19:48:55.038192 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.038161    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:55.086180 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.086148    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f-maas-api-service-ca\") pod \"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f\" (UID: \"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f\") "
Jul 23 19:48:55.086515 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.086493    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ea6249d-4ed9-4504-8cc7-fa0e2af8870f" (UID: "6ea6249d-4ed9-4504-8cc7-fa0e2af8870f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:48:55.187823 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.187742    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:48:55.906634 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.906604    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"
Jul 23 19:48:55.906826 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.906639    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm" event={"ID":"6ea6249d-4ed9-4504-8cc7-fa0e2af8870f","Type":"ContainerDied","Data":"bcd51ea52af983cf82bf0258b12bb40acc5fc4f27dea9582f70c3de8eb274a9f"}
Jul 23 19:48:55.906826 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:48:55.906697    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcd51ea52af983cf82bf0258b12bb40acc5fc4f27dea9582f70c3de8eb274a9f"
Jul 23 19:49:07.081814 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.081782    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"]
Jul 23 19:49:07.084369 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.082008    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh" podUID="f8a79cfb-c0d0-4429-abdb-b0994b920f6f" containerName="maas-api" containerID="cri-o://3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a" gracePeriod=30
Jul 23 19:49:07.332476 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.332420    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:49:07.502826 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.502796    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l86qg\" (UniqueName: \"kubernetes.io/projected/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-kube-api-access-l86qg\") pod \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") "
Jul 23 19:49:07.502983 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.502846    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls\") pod \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") "
Jul 23 19:49:07.502983 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.502893    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-openshift-service-ca\") pod \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\" (UID: \"f8a79cfb-c0d0-4429-abdb-b0994b920f6f\") "
Jul 23 19:49:07.503284 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.503259    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8a79cfb-c0d0-4429-abdb-b0994b920f6f" (UID: "f8a79cfb-c0d0-4429-abdb-b0994b920f6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:49:07.505074 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.505047    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8a79cfb-c0d0-4429-abdb-b0994b920f6f" (UID: "f8a79cfb-c0d0-4429-abdb-b0994b920f6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:49:07.505235 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.505050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-kube-api-access-l86qg" (OuterVolumeSpecName: "kube-api-access-l86qg") pod "f8a79cfb-c0d0-4429-abdb-b0994b920f6f" (UID: "f8a79cfb-c0d0-4429-abdb-b0994b920f6f"). InnerVolumeSpecName "kube-api-access-l86qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:49:07.604243 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.604155    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l86qg\" (UniqueName: \"kubernetes.io/projected/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-kube-api-access-l86qg\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:49:07.604243 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.604183    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:49:07.604243 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.604192    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a79cfb-c0d0-4429-abdb-b0994b920f6f-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:49:07.975382 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.975346    2566 generic.go:356] "Generic (PLEG): container finished" podID="f8a79cfb-c0d0-4429-abdb-b0994b920f6f" containerID="3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a" exitCode=0
Jul 23 19:49:07.975554 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.975403    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"
Jul 23 19:49:07.975554 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.975432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh" event={"ID":"f8a79cfb-c0d0-4429-abdb-b0994b920f6f","Type":"ContainerDied","Data":"3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a"}
Jul 23 19:49:07.975554 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.975502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh" event={"ID":"f8a79cfb-c0d0-4429-abdb-b0994b920f6f","Type":"ContainerDied","Data":"b3279c564269cced6d863e53d9b071995c789fcb9d692941e66f4dcc4083b460"}
Jul 23 19:49:07.975554 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.975523    2566 scope.go:117] "RemoveContainer" containerID="3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a"
Jul 23 19:49:07.984958 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.984944    2566 scope.go:117] "RemoveContainer" containerID="3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a"
Jul 23 19:49:07.985183 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:49:07.985163    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a\": container with ID starting with 3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a not found: ID does not exist" containerID="3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a"
Jul 23 19:49:07.985247 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.985194    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a"} err="failed to get container status \"3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a\": rpc error: code = NotFound desc = could not find container \"3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a\": container with ID starting with 3b1ef69c7022974749e136170e3add7d999f75f95e427cd1b62a6ea8fe0bef6a not found: ID does not exist"
Jul 23 19:49:07.997822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:07.997800    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"]
Jul 23 19:49:08.002781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:08.002761    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f275417d-797b59845-n8jjh"]
Jul 23 19:49:09.239017 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:09.238986    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a79cfb-c0d0-4429-abdb-b0994b920f6f" path="/var/lib/kubelet/pods/f8a79cfb-c0d0-4429-abdb-b0994b920f6f/volumes"
Jul 23 19:49:11.649518 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:11.649480    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"]
Jul 23 19:49:11.651994 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:11.651968    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f275417d-4rhkm"]
Jul 23 19:49:13.247732 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:13.247695    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea6249d-4ed9-4504-8cc7-fa0e2af8870f" path="/var/lib/kubelet/pods/6ea6249d-4ed9-4504-8cc7-fa0e2af8870f/volumes"
Jul 23 19:49:25.345261 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:49:25.345227    2566 scope.go:117] "RemoveContainer" containerID="765fa43da1c3118ea22c9fa9b0e693ab6e4cfe4cf10d71984ea4ba22ee857a36"
Jul 23 19:50:25.375194 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:25.375158    2566 scope.go:117] "RemoveContainer" containerID="da7a49a9cf5e36cbc40377d08a4c4ce5225042ca13eeb89bc4b3a1c34d159810"
Jul 23 19:50:59.764634 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.764546    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"]
Jul 23 19:50:59.765295 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.765270    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a79cfb-c0d0-4429-abdb-b0994b920f6f" containerName="maas-api"
Jul 23 19:50:59.765384 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.765298    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a79cfb-c0d0-4429-abdb-b0994b920f6f" containerName="maas-api"
Jul 23 19:50:59.765384 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.765343    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea6249d-4ed9-4504-8cc7-fa0e2af8870f" containerName="revoke-keys"
Jul 23 19:50:59.765384 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.765352    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea6249d-4ed9-4504-8cc7-fa0e2af8870f" containerName="revoke-keys"
Jul 23 19:50:59.765486 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.765423    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea6249d-4ed9-4504-8cc7-fa0e2af8870f" containerName="revoke-keys"
Jul 23 19:50:59.765486 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.765434    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a79cfb-c0d0-4429-abdb-b0994b920f6f" containerName="maas-api"
Jul 23 19:50:59.768446 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.768427    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.771174 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.771153    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:50:59.771295 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.771173    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:50:59.772106 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.772084    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:50:59.772192 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.772105    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-52a527-serving-cert\""
Jul 23 19:50:59.779324 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.779276    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"]
Jul 23 19:50:59.850580 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.850540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-maas-api-tls\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.850781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.850615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.850781 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.850693    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mtph\" (UniqueName: \"kubernetes.io/projected/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-kube-api-access-8mtph\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.951684 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.951644    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-maas-api-tls\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.951889 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.951733    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.951889 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.951760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mtph\" (UniqueName: \"kubernetes.io/projected/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-kube-api-access-8mtph\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.952389 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.952366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.954311 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.954288    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-maas-api-tls\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:50:59.963624 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:50:59.963600    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mtph\" (UniqueName: \"kubernetes.io/projected/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-kube-api-access-8mtph\") pod \"maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:51:00.080701 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:00.080594    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:51:00.224433 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:00.224406    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"]
Jul 23 19:51:00.226165 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:51:00.226126    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1e7103_0151_49ec_82a1_3fb3dcd07f5a.slice/crio-b1f063437289cbdf6f9e733ffd8063529b7d06361c82b9ed1ad030cb8f0cf97b WatchSource:0}: Error finding container b1f063437289cbdf6f9e733ffd8063529b7d06361c82b9ed1ad030cb8f0cf97b: Status 404 returned error can't find the container with id b1f063437289cbdf6f9e733ffd8063529b7d06361c82b9ed1ad030cb8f0cf97b
Jul 23 19:51:00.543742 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:00.543702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm" event={"ID":"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a","Type":"ContainerStarted","Data":"b1f063437289cbdf6f9e733ffd8063529b7d06361c82b9ed1ad030cb8f0cf97b"}
Jul 23 19:51:01.550700 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:01.550650    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm" event={"ID":"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a","Type":"ContainerStarted","Data":"0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db"}
Jul 23 19:51:01.551148 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:01.550797    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:51:01.570245 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:01.570194    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm" podStartSLOduration=2.126032097 podStartE2EDuration="2.570179129s" totalImagesPullingTime="444.147032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:51:01.568592795 +0000 UTC m=+2076.857192427" watchObservedRunningTime="2026-07-23 19:51:01.570179129 +0000 UTC m=+2076.858778755"
Jul 23 19:51:07.561081 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:07.561050    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:51:08.064979 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.064946    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"]
Jul 23 19:51:08.068522 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.068507    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.073091 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.073072    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ef2e0a-serving-cert\""
Jul 23 19:51:08.102978 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.102951    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"]
Jul 23 19:51:08.237664 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.237629    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1457a56-9dc5-4ca9-8995-d3103baeea4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.237844 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.237692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1457a56-9dc5-4ca9-8995-d3103baeea4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.237844 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.237772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmrmt\" (UniqueName: \"kubernetes.io/projected/c1457a56-9dc5-4ca9-8995-d3103baeea4f-kube-api-access-zmrmt\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.339072 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.338980    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1457a56-9dc5-4ca9-8995-d3103baeea4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.339072 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.339020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1457a56-9dc5-4ca9-8995-d3103baeea4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.339072 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.339060    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmrmt\" (UniqueName: \"kubernetes.io/projected/c1457a56-9dc5-4ca9-8995-d3103baeea4f-kube-api-access-zmrmt\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.339717 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.339657    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1457a56-9dc5-4ca9-8995-d3103baeea4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.341611 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.341592    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1457a56-9dc5-4ca9-8995-d3103baeea4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.346923 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.346902    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmrmt\" (UniqueName: \"kubernetes.io/projected/c1457a56-9dc5-4ca9-8995-d3103baeea4f-kube-api-access-zmrmt\") pod \"maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.378957 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.378932    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:08.516491 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.516464    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"]
Jul 23 19:51:08.519265 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:51:08.519228    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1457a56_9dc5_4ca9_8995_d3103baeea4f.slice/crio-c2d62bdcd214e3b8eabb4c89d8655606c530f3079158956a7ba0f6dac862e01b WatchSource:0}: Error finding container c2d62bdcd214e3b8eabb4c89d8655606c530f3079158956a7ba0f6dac862e01b: Status 404 returned error can't find the container with id c2d62bdcd214e3b8eabb4c89d8655606c530f3079158956a7ba0f6dac862e01b
Jul 23 19:51:08.585469 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:08.585438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" event={"ID":"c1457a56-9dc5-4ca9-8995-d3103baeea4f","Type":"ContainerStarted","Data":"c2d62bdcd214e3b8eabb4c89d8655606c530f3079158956a7ba0f6dac862e01b"}
Jul 23 19:51:09.592256 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:09.592221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" event={"ID":"c1457a56-9dc5-4ca9-8995-d3103baeea4f","Type":"ContainerStarted","Data":"84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5"}
Jul 23 19:51:09.592639 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:09.592363    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:09.609076 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:09.609027    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" podStartSLOduration=1.286149087 podStartE2EDuration="1.609013344s" totalImagesPullingTime="322.864257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:51:09.607344776 +0000 UTC m=+2084.895944367" watchObservedRunningTime="2026-07-23 19:51:09.609013344 +0000 UTC m=+2084.897612934"
Jul 23 19:51:15.602454 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:15.602426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:51:25.397873 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:25.397843    2566 scope.go:117] "RemoveContainer" containerID="1675efca7d80c1550b9695cf2bcc99f9c747237d575d170d43321a50ce63ce90"
Jul 23 19:51:25.406422 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:25.406404    2566 scope.go:117] "RemoveContainer" containerID="085484c1ab7f1ec0e7dc8cce9d0e4bd34ac13b997235daf088e4a29726299831"
Jul 23 19:51:25.414061 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:51:25.414042    2566 scope.go:117] "RemoveContainer" containerID="c16ecc4a128fcf8f38cff82be79e2f2580133e82d9c41224d0919f705e3a395f"
Jul 23 19:52:32.855761 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:32.855729    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"]
Jul 23 19:52:32.859118 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:32.859102    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:32.861494 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:32.861472    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:52:32.866802 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:32.866777    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"]
Jul 23 19:52:32.950802 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:32.950760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/945d2c51-ce99-4e7c-8797-25f35bbe2737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-52a527-dzt56\" (UID: \"945d2c51-ce99-4e7c-8797-25f35bbe2737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:33.051586 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:33.051554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/945d2c51-ce99-4e7c-8797-25f35bbe2737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-52a527-dzt56\" (UID: \"945d2c51-ce99-4e7c-8797-25f35bbe2737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:33.052193 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:33.052169    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/945d2c51-ce99-4e7c-8797-25f35bbe2737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-52a527-dzt56\" (UID: \"945d2c51-ce99-4e7c-8797-25f35bbe2737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:33.170017 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:33.169983    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:33.305885 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:33.305855    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"]
Jul 23 19:52:33.308838 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:52:33.308816    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945d2c51_ce99_4e7c_8797_25f35bbe2737.slice/crio-8402cbc1c448c6ac78b92120851d199d63796cc977a1211f8793ee32e704908d WatchSource:0}: Error finding container 8402cbc1c448c6ac78b92120851d199d63796cc977a1211f8793ee32e704908d: Status 404 returned error can't find the container with id 8402cbc1c448c6ac78b92120851d199d63796cc977a1211f8793ee32e704908d
Jul 23 19:52:34.024631 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:34.024599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56" event={"ID":"945d2c51-ce99-4e7c-8797-25f35bbe2737","Type":"ContainerStarted","Data":"7b4ab32001c620df2b69ab4038df1413cf654d37cf37c17d04eb3fb3549c96a2"}
Jul 23 19:52:34.024954 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:34.024665    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56" event={"ID":"945d2c51-ce99-4e7c-8797-25f35bbe2737","Type":"ContainerStarted","Data":"8402cbc1c448c6ac78b92120851d199d63796cc977a1211f8793ee32e704908d"}
Jul 23 19:52:35.031447 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:35.031415    2566 generic.go:356] "Generic (PLEG): container finished" podID="945d2c51-ce99-4e7c-8797-25f35bbe2737" containerID="7b4ab32001c620df2b69ab4038df1413cf654d37cf37c17d04eb3fb3549c96a2" exitCode=0
Jul 23 19:52:35.031827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:35.031477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56" event={"ID":"945d2c51-ce99-4e7c-8797-25f35bbe2737","Type":"ContainerDied","Data":"7b4ab32001c620df2b69ab4038df1413cf654d37cf37c17d04eb3fb3549c96a2"}
Jul 23 19:52:35.166539 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:35.166514    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:35.271967 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:35.271939    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/945d2c51-ce99-4e7c-8797-25f35bbe2737-maas-api-service-ca\") pod \"945d2c51-ce99-4e7c-8797-25f35bbe2737\" (UID: \"945d2c51-ce99-4e7c-8797-25f35bbe2737\") "
Jul 23 19:52:35.272318 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:35.272296    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/945d2c51-ce99-4e7c-8797-25f35bbe2737-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "945d2c51-ce99-4e7c-8797-25f35bbe2737" (UID: "945d2c51-ce99-4e7c-8797-25f35bbe2737"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:52:35.373227 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:35.373148    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/945d2c51-ce99-4e7c-8797-25f35bbe2737-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:52:36.037285 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:36.037257    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"
Jul 23 19:52:36.037659 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:36.037301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56" event={"ID":"945d2c51-ce99-4e7c-8797-25f35bbe2737","Type":"ContainerDied","Data":"8402cbc1c448c6ac78b92120851d199d63796cc977a1211f8793ee32e704908d"}
Jul 23 19:52:36.037659 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:36.037348    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8402cbc1c448c6ac78b92120851d199d63796cc977a1211f8793ee32e704908d"
Jul 23 19:52:43.452873 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.452838    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"]
Jul 23 19:52:43.453720 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.453612    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm" podUID="3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" containerName="maas-api" containerID="cri-o://0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db" gracePeriod=30
Jul 23 19:52:43.904862 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.904835    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:52:43.948272 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.948243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-openshift-service-ca\") pod \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") "
Jul 23 19:52:43.948438 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.948318    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mtph\" (UniqueName: \"kubernetes.io/projected/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-kube-api-access-8mtph\") pod \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") "
Jul 23 19:52:43.948438 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.948433    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-maas-api-tls\") pod \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\" (UID: \"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a\") "
Jul 23 19:52:43.948576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.948543    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" (UID: "3a1e7103-0151-49ec-82a1-3fb3dcd07f5a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:52:43.948757 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.948739    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:52:43.950576 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.950553    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" (UID: "3a1e7103-0151-49ec-82a1-3fb3dcd07f5a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:52:43.950740 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:43.950641    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-kube-api-access-8mtph" (OuterVolumeSpecName: "kube-api-access-8mtph") pod "3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" (UID: "3a1e7103-0151-49ec-82a1-3fb3dcd07f5a"). InnerVolumeSpecName "kube-api-access-8mtph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:52:44.049860 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.049785    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:52:44.049860 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.049813    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mtph\" (UniqueName: \"kubernetes.io/projected/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a-kube-api-access-8mtph\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:52:44.077190 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.077159    2566 generic.go:356] "Generic (PLEG): container finished" podID="3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" containerID="0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db" exitCode=0
Jul 23 19:52:44.077316 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.077201    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"
Jul 23 19:52:44.077316 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.077244    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm" event={"ID":"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a","Type":"ContainerDied","Data":"0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db"}
Jul 23 19:52:44.077316 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.077289    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm" event={"ID":"3a1e7103-0151-49ec-82a1-3fb3dcd07f5a","Type":"ContainerDied","Data":"b1f063437289cbdf6f9e733ffd8063529b7d06361c82b9ed1ad030cb8f0cf97b"}
Jul 23 19:52:44.077316 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.077311    2566 scope.go:117] "RemoveContainer" containerID="0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db"
Jul 23 19:52:44.086553 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.086537    2566 scope.go:117] "RemoveContainer" containerID="0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db"
Jul 23 19:52:44.086809 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:52:44.086792    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db\": container with ID starting with 0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db not found: ID does not exist" containerID="0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db"
Jul 23 19:52:44.086871 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.086818    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db"} err="failed to get container status \"0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db\": rpc error: code = NotFound desc = could not find container \"0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db\": container with ID starting with 0147a7f1808e1255158dce47264e60bc21392e18b1a374466954025012cae1db not found: ID does not exist"
Jul 23 19:52:44.099462 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.099436    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"]
Jul 23 19:52:44.102827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:44.102807    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-52a527-f5fc64789-zmqzm"]
Jul 23 19:52:45.238808 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:45.238774    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" path="/var/lib/kubelet/pods/3a1e7103-0151-49ec-82a1-3fb3dcd07f5a/volumes"
Jul 23 19:52:47.942842 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:47.942799    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"]
Jul 23 19:52:47.944634 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:47.944612    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a527-dzt56"]
Jul 23 19:52:49.238488 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:49.238449    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945d2c51-ce99-4e7c-8797-25f35bbe2737" path="/var/lib/kubelet/pods/945d2c51-ce99-4e7c-8797-25f35bbe2737/volumes"
Jul 23 19:52:54.853611 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.853576    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"]
Jul 23 19:52:54.854060 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.853972    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945d2c51-ce99-4e7c-8797-25f35bbe2737" containerName="revoke-keys"
Jul 23 19:52:54.854060 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.853984    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="945d2c51-ce99-4e7c-8797-25f35bbe2737" containerName="revoke-keys"
Jul 23 19:52:54.854060 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.854013    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" containerName="maas-api"
Jul 23 19:52:54.854060 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.854019    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" containerName="maas-api"
Jul 23 19:52:54.854208 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.854073    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="945d2c51-ce99-4e7c-8797-25f35bbe2737" containerName="revoke-keys"
Jul 23 19:52:54.854208 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.854083    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a1e7103-0151-49ec-82a1-3fb3dcd07f5a" containerName="maas-api"
Jul 23 19:52:54.858705 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.858660    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:54.861341 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.861314    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:52:54.865058 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.865036    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"]
Jul 23 19:52:54.951401 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:54.951352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b782eaa2-0844-4ce9-8f87-ea88c25a8ada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl\" (UID: \"b782eaa2-0844-4ce9-8f87-ea88c25a8ada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:55.052295 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:55.052254    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b782eaa2-0844-4ce9-8f87-ea88c25a8ada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl\" (UID: \"b782eaa2-0844-4ce9-8f87-ea88c25a8ada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:55.052924 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:55.052903    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b782eaa2-0844-4ce9-8f87-ea88c25a8ada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl\" (UID: \"b782eaa2-0844-4ce9-8f87-ea88c25a8ada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:55.171387 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:55.171356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:55.307869 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:55.307634    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"]
Jul 23 19:52:55.310147 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:52:55.310119    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb782eaa2_0844_4ce9_8f87_ea88c25a8ada.slice/crio-1fad42c0acbb4a41156ef5d2f7a1191a46ef7b26083bc0be684076f095833f80 WatchSource:0}: Error finding container 1fad42c0acbb4a41156ef5d2f7a1191a46ef7b26083bc0be684076f095833f80: Status 404 returned error can't find the container with id 1fad42c0acbb4a41156ef5d2f7a1191a46ef7b26083bc0be684076f095833f80
Jul 23 19:52:56.142216 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:56.142184    2566 generic.go:356] "Generic (PLEG): container finished" podID="b782eaa2-0844-4ce9-8f87-ea88c25a8ada" containerID="6d61d62eecca1d36d94f3a296424efb6a7b8b1add67dbcf05a7d6f42bfb15f85" exitCode=0
Jul 23 19:52:56.142619 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:56.142266    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl" event={"ID":"b782eaa2-0844-4ce9-8f87-ea88c25a8ada","Type":"ContainerDied","Data":"6d61d62eecca1d36d94f3a296424efb6a7b8b1add67dbcf05a7d6f42bfb15f85"}
Jul 23 19:52:56.142619 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:56.142331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl" event={"ID":"b782eaa2-0844-4ce9-8f87-ea88c25a8ada","Type":"ContainerStarted","Data":"1fad42c0acbb4a41156ef5d2f7a1191a46ef7b26083bc0be684076f095833f80"}
Jul 23 19:52:57.288399 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:57.288373    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:57.375816 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:57.375783    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b782eaa2-0844-4ce9-8f87-ea88c25a8ada-maas-api-service-ca\") pod \"b782eaa2-0844-4ce9-8f87-ea88c25a8ada\" (UID: \"b782eaa2-0844-4ce9-8f87-ea88c25a8ada\") "
Jul 23 19:52:57.376164 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:57.376137    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b782eaa2-0844-4ce9-8f87-ea88c25a8ada-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b782eaa2-0844-4ce9-8f87-ea88c25a8ada" (UID: "b782eaa2-0844-4ce9-8f87-ea88c25a8ada"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:52:57.477150 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:57.477067    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b782eaa2-0844-4ce9-8f87-ea88c25a8ada-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:52:58.156649 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:58.156623    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"
Jul 23 19:52:58.156847 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:58.156621    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl" event={"ID":"b782eaa2-0844-4ce9-8f87-ea88c25a8ada","Type":"ContainerDied","Data":"1fad42c0acbb4a41156ef5d2f7a1191a46ef7b26083bc0be684076f095833f80"}
Jul 23 19:52:58.156847 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:52:58.156751    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fad42c0acbb4a41156ef5d2f7a1191a46ef7b26083bc0be684076f095833f80"
Jul 23 19:53:05.408011 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.407967    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"]
Jul 23 19:53:05.408572 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.408175    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" podUID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" containerName="maas-api" containerID="cri-o://84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5" gracePeriod=30
Jul 23 19:53:05.654350 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.654327    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:53:05.759513 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.759425    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1457a56-9dc5-4ca9-8995-d3103baeea4f-maas-api-tls\") pod \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") "
Jul 23 19:53:05.759513 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.759465    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1457a56-9dc5-4ca9-8995-d3103baeea4f-openshift-service-ca\") pod \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") "
Jul 23 19:53:05.759513 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.759497    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmrmt\" (UniqueName: \"kubernetes.io/projected/c1457a56-9dc5-4ca9-8995-d3103baeea4f-kube-api-access-zmrmt\") pod \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\" (UID: \"c1457a56-9dc5-4ca9-8995-d3103baeea4f\") "
Jul 23 19:53:05.759920 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.759895    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1457a56-9dc5-4ca9-8995-d3103baeea4f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1457a56-9dc5-4ca9-8995-d3103baeea4f" (UID: "c1457a56-9dc5-4ca9-8995-d3103baeea4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:53:05.761846 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.761818    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1457a56-9dc5-4ca9-8995-d3103baeea4f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1457a56-9dc5-4ca9-8995-d3103baeea4f" (UID: "c1457a56-9dc5-4ca9-8995-d3103baeea4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:53:05.761846 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.761824    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1457a56-9dc5-4ca9-8995-d3103baeea4f-kube-api-access-zmrmt" (OuterVolumeSpecName: "kube-api-access-zmrmt") pod "c1457a56-9dc5-4ca9-8995-d3103baeea4f" (UID: "c1457a56-9dc5-4ca9-8995-d3103baeea4f"). InnerVolumeSpecName "kube-api-access-zmrmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:53:05.860657 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.860623    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1457a56-9dc5-4ca9-8995-d3103baeea4f-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:53:05.860657 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.860652    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1457a56-9dc5-4ca9-8995-d3103baeea4f-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:53:05.860657 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:05.860663    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmrmt\" (UniqueName: \"kubernetes.io/projected/c1457a56-9dc5-4ca9-8995-d3103baeea4f-kube-api-access-zmrmt\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:53:06.198664 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.198629    2566 generic.go:356] "Generic (PLEG): container finished" podID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" containerID="84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5" exitCode=0
Jul 23 19:53:06.198930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.198699    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"
Jul 23 19:53:06.198930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.198716    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" event={"ID":"c1457a56-9dc5-4ca9-8995-d3103baeea4f","Type":"ContainerDied","Data":"84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5"}
Jul 23 19:53:06.198930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.198767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" event={"ID":"c1457a56-9dc5-4ca9-8995-d3103baeea4f","Type":"ContainerDied","Data":"c2d62bdcd214e3b8eabb4c89d8655606c530f3079158956a7ba0f6dac862e01b"}
Jul 23 19:53:06.198930 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.198790    2566 scope.go:117] "RemoveContainer" containerID="84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5"
Jul 23 19:53:06.208594 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.208577    2566 scope.go:117] "RemoveContainer" containerID="84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5"
Jul 23 19:53:06.208869 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:53:06.208850    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5\": container with ID starting with 84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5 not found: ID does not exist" containerID="84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5"
Jul 23 19:53:06.208917 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.208877    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5"} err="failed to get container status \"84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5\": rpc error: code = NotFound desc = could not find container \"84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5\": container with ID starting with 84808afe477d4eac3d6b6f85597ad6757c1a4d0504067b18275875c4ecd667d5 not found: ID does not exist"
Jul 23 19:53:06.220762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.220739    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"]
Jul 23 19:53:06.222536 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:06.222516    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt"]
Jul 23 19:53:07.238526 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:07.238488    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" path="/var/lib/kubelet/pods/c1457a56-9dc5-4ca9-8995-d3103baeea4f/volumes"
Jul 23 19:53:08.598834 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:08.598771    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef2e0a-dddc4fd9-bn4tt" podUID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.50:8443/health\": context deadline exceeded"
Jul 23 19:53:09.942619 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:09.942582    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"]
Jul 23 19:53:09.944573 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:09.944543    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef2e0a-s6lpl"]
Jul 23 19:53:11.239065 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:11.239023    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b782eaa2-0844-4ce9-8f87-ea88c25a8ada" path="/var/lib/kubelet/pods/b782eaa2-0844-4ce9-8f87-ea88c25a8ada/volumes"
Jul 23 19:53:20.585947 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.585864    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"]
Jul 23 19:53:20.586419 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.586241    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b782eaa2-0844-4ce9-8f87-ea88c25a8ada" containerName="revoke-keys"
Jul 23 19:53:20.586419 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.586252    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b782eaa2-0844-4ce9-8f87-ea88c25a8ada" containerName="revoke-keys"
Jul 23 19:53:20.586419 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.586276    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" containerName="maas-api"
Jul 23 19:53:20.586419 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.586281    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" containerName="maas-api"
Jul 23 19:53:20.586419 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.586342    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1457a56-9dc5-4ca9-8995-d3103baeea4f" containerName="maas-api"
Jul 23 19:53:20.586419 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.586350    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b782eaa2-0844-4ce9-8f87-ea88c25a8ada" containerName="revoke-keys"
Jul 23 19:53:20.590813 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.590788    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.593272 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.593248    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-081c5e-serving-cert\""
Jul 23 19:53:20.593381 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.593305    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:53:20.593381 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.593305    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:53:20.594042 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.594019    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:53:20.598414 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.598393    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"]
Jul 23 19:53:20.691827 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.691790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr7sx\" (UniqueName: \"kubernetes.io/projected/ee4a4a99-a239-4055-ab44-b1e469b6f666-kube-api-access-mr7sx\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.692006 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.691842    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee4a4a99-a239-4055-ab44-b1e469b6f666-openshift-service-ca\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.692006 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.691905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee4a4a99-a239-4055-ab44-b1e469b6f666-maas-api-tls\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.793382 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.793346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr7sx\" (UniqueName: \"kubernetes.io/projected/ee4a4a99-a239-4055-ab44-b1e469b6f666-kube-api-access-mr7sx\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.793573 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.793406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee4a4a99-a239-4055-ab44-b1e469b6f666-openshift-service-ca\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.793573 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.793444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee4a4a99-a239-4055-ab44-b1e469b6f666-maas-api-tls\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.794177 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.794153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee4a4a99-a239-4055-ab44-b1e469b6f666-openshift-service-ca\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.796158 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.796134    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee4a4a99-a239-4055-ab44-b1e469b6f666-maas-api-tls\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.801407 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.801384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr7sx\" (UniqueName: \"kubernetes.io/projected/ee4a4a99-a239-4055-ab44-b1e469b6f666-kube-api-access-mr7sx\") pod \"maas-api-e2e-api-a-081c5e-f548b445b-lvq9s\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:20.904157 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:20.904135    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:21.044349 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:21.044324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"]
Jul 23 19:53:21.046606 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:53:21.046574    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4a4a99_a239_4055_ab44_b1e469b6f666.slice/crio-2e564da0135479c3877614f9556bdaf0c3ae924752183b6805979d7b6f05c349 WatchSource:0}: Error finding container 2e564da0135479c3877614f9556bdaf0c3ae924752183b6805979d7b6f05c349: Status 404 returned error can't find the container with id 2e564da0135479c3877614f9556bdaf0c3ae924752183b6805979d7b6f05c349
Jul 23 19:53:21.048073 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:21.048053    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 19:53:21.275992 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:21.275903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s" event={"ID":"ee4a4a99-a239-4055-ab44-b1e469b6f666","Type":"ContainerStarted","Data":"2e564da0135479c3877614f9556bdaf0c3ae924752183b6805979d7b6f05c349"}
Jul 23 19:53:22.282572 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:22.282532    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s" event={"ID":"ee4a4a99-a239-4055-ab44-b1e469b6f666","Type":"ContainerStarted","Data":"4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9"}
Jul 23 19:53:22.282968 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:22.282633    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:22.300010 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:22.299964    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s" podStartSLOduration=1.797268039 podStartE2EDuration="2.299949355s" totalImagesPullingTime="502.681316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:53:22.298238826 +0000 UTC m=+2217.586838419" watchObservedRunningTime="2026-07-23 19:53:22.299949355 +0000 UTC m=+2217.588548945"
Jul 23 19:53:25.450973 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:25.450937    2566 scope.go:117] "RemoveContainer" containerID="25b48e51420a3d9ff894a70a56a34df1948a0ac7fa59ee5021fa78e53b2b0267"
Jul 23 19:53:28.295584 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.295556    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:28.914335 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.914301    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"]
Jul 23 19:53:28.917940 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.917915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:28.920358 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.920338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-dbc659-serving-cert\""
Jul 23 19:53:28.928257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.928233    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"]
Jul 23 19:53:28.965886 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.965858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53689d78-3a84-4286-90fe-538873fd6c98-openshift-service-ca\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:28.966048 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.965928    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr69h\" (UniqueName: \"kubernetes.io/projected/53689d78-3a84-4286-90fe-538873fd6c98-kube-api-access-gr69h\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:28.966048 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:28.965974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53689d78-3a84-4286-90fe-538873fd6c98-maas-api-tls\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.067521 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.067485    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr69h\" (UniqueName: \"kubernetes.io/projected/53689d78-3a84-4286-90fe-538873fd6c98-kube-api-access-gr69h\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.067817 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.067797    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53689d78-3a84-4286-90fe-538873fd6c98-maas-api-tls\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.068018 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.068004    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53689d78-3a84-4286-90fe-538873fd6c98-openshift-service-ca\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.068905 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.068881    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53689d78-3a84-4286-90fe-538873fd6c98-openshift-service-ca\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.071771 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.071749    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53689d78-3a84-4286-90fe-538873fd6c98-maas-api-tls\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.077746 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.077726    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr69h\" (UniqueName: \"kubernetes.io/projected/53689d78-3a84-4286-90fe-538873fd6c98-kube-api-access-gr69h\") pod \"maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.230496 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.230331    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:29.403819 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:29.403794    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"]
Jul 23 19:53:29.405901 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:53:29.405869    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53689d78_3a84_4286_90fe_538873fd6c98.slice/crio-820e7373971cb7c8c3d047fcac4ef00aa6dad17d5bc672d8d692bd9ebcad1a61 WatchSource:0}: Error finding container 820e7373971cb7c8c3d047fcac4ef00aa6dad17d5bc672d8d692bd9ebcad1a61: Status 404 returned error can't find the container with id 820e7373971cb7c8c3d047fcac4ef00aa6dad17d5bc672d8d692bd9ebcad1a61
Jul 23 19:53:30.328911 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:30.328875    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq" event={"ID":"53689d78-3a84-4286-90fe-538873fd6c98","Type":"ContainerStarted","Data":"c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23"}
Jul 23 19:53:30.329112 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:30.328932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq" event={"ID":"53689d78-3a84-4286-90fe-538873fd6c98","Type":"ContainerStarted","Data":"820e7373971cb7c8c3d047fcac4ef00aa6dad17d5bc672d8d692bd9ebcad1a61"}
Jul 23 19:53:30.329112 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:30.328984    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:30.350065 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:30.350018    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq" podStartSLOduration=1.982246005 podStartE2EDuration="2.350004749s" totalImagesPullingTime="367.758744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:53:30.348322261 +0000 UTC m=+2225.636921870" watchObservedRunningTime="2026-07-23 19:53:30.350004749 +0000 UTC m=+2225.638604340"
Jul 23 19:53:36.339820 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:36.339795    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:53:56.187119 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.187024    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"]
Jul 23 19:53:56.187593 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.187266    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s" podUID="ee4a4a99-a239-4055-ab44-b1e469b6f666" containerName="maas-api" containerID="cri-o://4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9" gracePeriod=30
Jul 23 19:53:56.437880 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.437819    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:56.463956 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.463925    2566 generic.go:356] "Generic (PLEG): container finished" podID="ee4a4a99-a239-4055-ab44-b1e469b6f666" containerID="4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9" exitCode=0
Jul 23 19:53:56.464092 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.464023    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"
Jul 23 19:53:56.464092 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.464043    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s" event={"ID":"ee4a4a99-a239-4055-ab44-b1e469b6f666","Type":"ContainerDied","Data":"4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9"}
Jul 23 19:53:56.464183 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.464098    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s" event={"ID":"ee4a4a99-a239-4055-ab44-b1e469b6f666","Type":"ContainerDied","Data":"2e564da0135479c3877614f9556bdaf0c3ae924752183b6805979d7b6f05c349"}
Jul 23 19:53:56.464183 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.464131    2566 scope.go:117] "RemoveContainer" containerID="4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9"
Jul 23 19:53:56.476574 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.476553    2566 scope.go:117] "RemoveContainer" containerID="4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9"
Jul 23 19:53:56.476858 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:53:56.476836    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9\": container with ID starting with 4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9 not found: ID does not exist" containerID="4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9"
Jul 23 19:53:56.476964 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.476867    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9"} err="failed to get container status \"4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9\": rpc error: code = NotFound desc = could not find container \"4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9\": container with ID starting with 4197d8d540bfc825db7ea6e1c971de12b2d7ecb138920cb76e5a58ed69f0c6d9 not found: ID does not exist"
Jul 23 19:53:56.531428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.531402    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee4a4a99-a239-4055-ab44-b1e469b6f666-openshift-service-ca\") pod \"ee4a4a99-a239-4055-ab44-b1e469b6f666\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") "
Jul 23 19:53:56.531428 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.531430    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr7sx\" (UniqueName: \"kubernetes.io/projected/ee4a4a99-a239-4055-ab44-b1e469b6f666-kube-api-access-mr7sx\") pod \"ee4a4a99-a239-4055-ab44-b1e469b6f666\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") "
Jul 23 19:53:56.531613 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.531526    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee4a4a99-a239-4055-ab44-b1e469b6f666-maas-api-tls\") pod \"ee4a4a99-a239-4055-ab44-b1e469b6f666\" (UID: \"ee4a4a99-a239-4055-ab44-b1e469b6f666\") "
Jul 23 19:53:56.531787 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.531761    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee4a4a99-a239-4055-ab44-b1e469b6f666-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee4a4a99-a239-4055-ab44-b1e469b6f666" (UID: "ee4a4a99-a239-4055-ab44-b1e469b6f666"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:53:56.533636 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.533614    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee4a4a99-a239-4055-ab44-b1e469b6f666-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee4a4a99-a239-4055-ab44-b1e469b6f666" (UID: "ee4a4a99-a239-4055-ab44-b1e469b6f666"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:53:56.533764 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.533702    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee4a4a99-a239-4055-ab44-b1e469b6f666-kube-api-access-mr7sx" (OuterVolumeSpecName: "kube-api-access-mr7sx") pod "ee4a4a99-a239-4055-ab44-b1e469b6f666" (UID: "ee4a4a99-a239-4055-ab44-b1e469b6f666"). InnerVolumeSpecName "kube-api-access-mr7sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:53:56.632203 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.632175    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee4a4a99-a239-4055-ab44-b1e469b6f666-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:53:56.632203 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.632201    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee4a4a99-a239-4055-ab44-b1e469b6f666-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:53:56.632420 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.632213    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr7sx\" (UniqueName: \"kubernetes.io/projected/ee4a4a99-a239-4055-ab44-b1e469b6f666-kube-api-access-mr7sx\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:53:56.787024 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.786994    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"]
Jul 23 19:53:56.791250 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:56.791225    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-081c5e-f548b445b-lvq9s"]
Jul 23 19:53:57.239100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:53:57.239069    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee4a4a99-a239-4055-ab44-b1e469b6f666" path="/var/lib/kubelet/pods/ee4a4a99-a239-4055-ab44-b1e469b6f666/volumes"
Jul 23 19:54:07.821835 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.821797    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"]
Jul 23 19:54:07.824413 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.822347    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee4a4a99-a239-4055-ab44-b1e469b6f666" containerName="maas-api"
Jul 23 19:54:07.824413 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.822363    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee4a4a99-a239-4055-ab44-b1e469b6f666" containerName="maas-api"
Jul 23 19:54:07.824413 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.822438    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee4a4a99-a239-4055-ab44-b1e469b6f666" containerName="maas-api"
Jul 23 19:54:07.825762 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.825743    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:07.828085 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.828066    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 19:54:07.832159 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.832137    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"]
Jul 23 19:54:07.935190 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:07.935150    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2\" (UID: \"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:08.036337 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:08.036304    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2\" (UID: \"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:08.037211 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:08.037170    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2\" (UID: \"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:08.137740 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:08.137619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:08.275261 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:08.275231    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"]
Jul 23 19:54:08.277398 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:54:08.277373    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9c9a57_0c45_4473_bbc9_4284f9fb3a09.slice/crio-048c08bb0452c88c7e4edec6cf2f8948f20686cae13f1b457508eca2deaabb57 WatchSource:0}: Error finding container 048c08bb0452c88c7e4edec6cf2f8948f20686cae13f1b457508eca2deaabb57: Status 404 returned error can't find the container with id 048c08bb0452c88c7e4edec6cf2f8948f20686cae13f1b457508eca2deaabb57
Jul 23 19:54:08.524522 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:08.524492    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2" event={"ID":"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09","Type":"ContainerStarted","Data":"048c08bb0452c88c7e4edec6cf2f8948f20686cae13f1b457508eca2deaabb57"}
Jul 23 19:54:09.530496 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:09.530459    2566 generic.go:356] "Generic (PLEG): container finished" podID="9e9c9a57-0c45-4473-bbc9-4284f9fb3a09" containerID="e017922fbe16200d30a8f33c74869252371a1c92ad05b423e966036549f4ba06" exitCode=0
Jul 23 19:54:09.530950 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:09.530555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2" event={"ID":"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09","Type":"ContainerDied","Data":"e017922fbe16200d30a8f33c74869252371a1c92ad05b423e966036549f4ba06"}
Jul 23 19:54:10.667467 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:10.667442    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:10.761184 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:10.761152    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09-maas-api-service-ca\") pod \"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09\" (UID: \"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09\") "
Jul 23 19:54:10.761529 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:10.761508    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e9c9a57-0c45-4473-bbc9-4284f9fb3a09" (UID: "9e9c9a57-0c45-4473-bbc9-4284f9fb3a09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:54:10.862246 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:10.862163    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:54:11.542506 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:11.542422    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"
Jul 23 19:54:11.542506 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:11.542466    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2" event={"ID":"9e9c9a57-0c45-4473-bbc9-4284f9fb3a09","Type":"ContainerDied","Data":"048c08bb0452c88c7e4edec6cf2f8948f20686cae13f1b457508eca2deaabb57"}
Jul 23 19:54:11.542506 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:11.542509    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="048c08bb0452c88c7e4edec6cf2f8948f20686cae13f1b457508eca2deaabb57"
Jul 23 19:54:23.252378 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.252343    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"]
Jul 23 19:54:23.252800 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.252554    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq" podUID="53689d78-3a84-4286-90fe-538873fd6c98" containerName="maas-api" containerID="cri-o://c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23" gracePeriod=30
Jul 23 19:54:23.508892 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.508833    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:54:23.583950 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.583911    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53689d78-3a84-4286-90fe-538873fd6c98-openshift-service-ca\") pod \"53689d78-3a84-4286-90fe-538873fd6c98\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") "
Jul 23 19:54:23.583950 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.583952    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr69h\" (UniqueName: \"kubernetes.io/projected/53689d78-3a84-4286-90fe-538873fd6c98-kube-api-access-gr69h\") pod \"53689d78-3a84-4286-90fe-538873fd6c98\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") "
Jul 23 19:54:23.584212 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.583976    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53689d78-3a84-4286-90fe-538873fd6c98-maas-api-tls\") pod \"53689d78-3a84-4286-90fe-538873fd6c98\" (UID: \"53689d78-3a84-4286-90fe-538873fd6c98\") "
Jul 23 19:54:23.584395 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.584367    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53689d78-3a84-4286-90fe-538873fd6c98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53689d78-3a84-4286-90fe-538873fd6c98" (UID: "53689d78-3a84-4286-90fe-538873fd6c98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 19:54:23.586276 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.586252    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53689d78-3a84-4286-90fe-538873fd6c98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53689d78-3a84-4286-90fe-538873fd6c98" (UID: "53689d78-3a84-4286-90fe-538873fd6c98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:54:23.586341 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.586256    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53689d78-3a84-4286-90fe-538873fd6c98-kube-api-access-gr69h" (OuterVolumeSpecName: "kube-api-access-gr69h") pod "53689d78-3a84-4286-90fe-538873fd6c98" (UID: "53689d78-3a84-4286-90fe-538873fd6c98"). InnerVolumeSpecName "kube-api-access-gr69h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:54:23.601997 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.601971    2566 generic.go:356] "Generic (PLEG): container finished" podID="53689d78-3a84-4286-90fe-538873fd6c98" containerID="c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23" exitCode=0
Jul 23 19:54:23.602105 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.602018    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"
Jul 23 19:54:23.602105 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.602060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq" event={"ID":"53689d78-3a84-4286-90fe-538873fd6c98","Type":"ContainerDied","Data":"c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23"}
Jul 23 19:54:23.602176 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.602122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq" event={"ID":"53689d78-3a84-4286-90fe-538873fd6c98","Type":"ContainerDied","Data":"820e7373971cb7c8c3d047fcac4ef00aa6dad17d5bc672d8d692bd9ebcad1a61"}
Jul 23 19:54:23.602176 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.602155    2566 scope.go:117] "RemoveContainer" containerID="c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23"
Jul 23 19:54:23.612184 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.612165    2566 scope.go:117] "RemoveContainer" containerID="c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23"
Jul 23 19:54:23.612479 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:54:23.612456    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23\": container with ID starting with c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23 not found: ID does not exist" containerID="c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23"
Jul 23 19:54:23.612558 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.612487    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23"} err="failed to get container status \"c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23\": rpc error: code = NotFound desc = could not find container \"c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23\": container with ID starting with c66f74815deffd53365b4bcb5c3fcda6f0511824b89df270ab66c2be3e907a23 not found: ID does not exist"
Jul 23 19:54:23.623154 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.623133    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"]
Jul 23 19:54:23.627100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.627080    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dbc659-7dd97856fb-6nrpq"]
Jul 23 19:54:23.685324 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.685297    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53689d78-3a84-4286-90fe-538873fd6c98-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:54:23.685324 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.685320    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53689d78-3a84-4286-90fe-538873fd6c98-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:54:23.685468 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:23.685330    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr69h\" (UniqueName: \"kubernetes.io/projected/53689d78-3a84-4286-90fe-538873fd6c98-kube-api-access-gr69h\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:54:25.239911 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:25.239882    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53689d78-3a84-4286-90fe-538873fd6c98" path="/var/lib/kubelet/pods/53689d78-3a84-4286-90fe-538873fd6c98/volumes"
Jul 23 19:54:27.902777 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:27.902735    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"]
Jul 23 19:54:27.904510 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:27.904485    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dbc659-cm2t2"]
Jul 23 19:54:29.238308 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:29.238273    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e9c9a57-0c45-4473-bbc9-4284f9fb3a09" path="/var/lib/kubelet/pods/9e9c9a57-0c45-4473-bbc9-4284f9fb3a09/volumes"
Jul 23 19:54:38.186577 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.186532    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"]
Jul 23 19:54:38.187041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.186973    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9c9a57-0c45-4473-bbc9-4284f9fb3a09" containerName="revoke-keys"
Jul 23 19:54:38.187041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.186985    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9c9a57-0c45-4473-bbc9-4284f9fb3a09" containerName="revoke-keys"
Jul 23 19:54:38.187041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.187000    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53689d78-3a84-4286-90fe-538873fd6c98" containerName="maas-api"
Jul 23 19:54:38.187041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.187008    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53689d78-3a84-4286-90fe-538873fd6c98" containerName="maas-api"
Jul 23 19:54:38.187170 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.187094    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e9c9a57-0c45-4473-bbc9-4284f9fb3a09" containerName="revoke-keys"
Jul 23 19:54:38.187170 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.187103    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53689d78-3a84-4286-90fe-538873fd6c98" containerName="maas-api"
Jul 23 19:54:38.190116 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.190097    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.192499 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.192474    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 19:54:38.192612 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.192501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-459a5a-serving-cert\""
Jul 23 19:54:38.193305 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.193287    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 19:54:38.193305 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.193297    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vk5pr\""
Jul 23 19:54:38.200916 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.200877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"]
Jul 23 19:54:38.324695 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.324644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5659ba-c43b-46be-a134-d39fb6245aed-maas-api-tls\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.324857 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.324732    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffcbg\" (UniqueName: \"kubernetes.io/projected/0e5659ba-c43b-46be-a134-d39fb6245aed-kube-api-access-ffcbg\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.324929 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.324911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5659ba-c43b-46be-a134-d39fb6245aed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.425472 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.425436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5659ba-c43b-46be-a134-d39fb6245aed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.425649 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.425505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5659ba-c43b-46be-a134-d39fb6245aed-maas-api-tls\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.425649 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.425629    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffcbg\" (UniqueName: \"kubernetes.io/projected/0e5659ba-c43b-46be-a134-d39fb6245aed-kube-api-access-ffcbg\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.426167 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.426151    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5659ba-c43b-46be-a134-d39fb6245aed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.428057 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.428036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5659ba-c43b-46be-a134-d39fb6245aed-maas-api-tls\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.433882 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.433859    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffcbg\" (UniqueName: \"kubernetes.io/projected/0e5659ba-c43b-46be-a134-d39fb6245aed-kube-api-access-ffcbg\") pod \"maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.502130 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.502042    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:38.649342 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.649132    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"]
Jul 23 19:54:38.651645 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:54:38.651617    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e5659ba_c43b_46be_a134_d39fb6245aed.slice/crio-f53b8850ca866a0fb2af1377589937cf6089f9421eced4d27c77bd93bcb2a1a7 WatchSource:0}: Error finding container f53b8850ca866a0fb2af1377589937cf6089f9421eced4d27c77bd93bcb2a1a7: Status 404 returned error can't find the container with id f53b8850ca866a0fb2af1377589937cf6089f9421eced4d27c77bd93bcb2a1a7
Jul 23 19:54:38.677506 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:38.677464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg" event={"ID":"0e5659ba-c43b-46be-a134-d39fb6245aed","Type":"ContainerStarted","Data":"f53b8850ca866a0fb2af1377589937cf6089f9421eced4d27c77bd93bcb2a1a7"}
Jul 23 19:54:39.684177 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:39.684145    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg" event={"ID":"0e5659ba-c43b-46be-a134-d39fb6245aed","Type":"ContainerStarted","Data":"0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa"}
Jul 23 19:54:39.684542 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:39.684199    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:39.715652 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:39.715602    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg" podStartSLOduration=1.343743876 podStartE2EDuration="1.715583289s" totalImagesPullingTime="371.839413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:54:39.713743555 +0000 UTC m=+2295.002343150" watchObservedRunningTime="2026-07-23 19:54:39.715583289 +0000 UTC m=+2295.004182879"
Jul 23 19:54:45.695511 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:45.695475    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 19:54:46.193463 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.193426    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"]
Jul 23 19:54:46.198063 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.198037    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.200529 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.200502    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-829079-serving-cert\""
Jul 23 19:54:46.200856 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.200833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.200958 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.200939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5x4j\" (UniqueName: \"kubernetes.io/projected/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-kube-api-access-l5x4j\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.201068 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.201047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-openshift-service-ca\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.208720 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.208553    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"]
Jul 23 19:54:46.301915 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.301885    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5x4j\" (UniqueName: \"kubernetes.io/projected/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-kube-api-access-l5x4j\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.302131 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.301960    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-openshift-service-ca\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.302131 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.302010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.302131 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:54:46.302107    2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-serving-cert: secret "maas-api-e2e-shared-b-829079-serving-cert" not found
Jul 23 19:54:46.302310 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:54:46.302166    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls podName:2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63 nodeName:}" failed. No retries permitted until 2026-07-23 19:54:46.802146124 +0000 UTC m=+2302.090745692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls") pod "maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" (UID: "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63") : secret "maas-api-e2e-shared-b-829079-serving-cert" not found
Jul 23 19:54:46.302577 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.302555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-openshift-service-ca\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.310683 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.310640    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5x4j\" (UniqueName: \"kubernetes.io/projected/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-kube-api-access-l5x4j\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.807443 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.807398    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.810194 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.810167    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls\") pod \"maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.810916 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.810887    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:46.951445 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:46.951242    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"]
Jul 23 19:54:46.953989 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:54:46.953960    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8a4945_34d5_4b6e_8a70_7ee42f4e2d63.slice/crio-74721c7d24f619be1fbf6871ac2fca7993e8252e61bc233fbbb6eb43e6a80917 WatchSource:0}: Error finding container 74721c7d24f619be1fbf6871ac2fca7993e8252e61bc233fbbb6eb43e6a80917: Status 404 returned error can't find the container with id 74721c7d24f619be1fbf6871ac2fca7993e8252e61bc233fbbb6eb43e6a80917
Jul 23 19:54:47.727223 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:47.727188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" event={"ID":"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63","Type":"ContainerStarted","Data":"762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea"}
Jul 23 19:54:47.727396 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:47.727240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" event={"ID":"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63","Type":"ContainerStarted","Data":"74721c7d24f619be1fbf6871ac2fca7993e8252e61bc233fbbb6eb43e6a80917"}
Jul 23 19:54:47.727396 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:47.727304    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:54:47.743911 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:47.743855    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" podStartSLOduration=1.393818151 podStartE2EDuration="1.743838946s" totalImagesPullingTime="350.020795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:54:47.742109899 +0000 UTC m=+2303.030709490" watchObservedRunningTime="2026-07-23 19:54:47.743838946 +0000 UTC m=+2303.032438541"
Jul 23 19:54:53.738498 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:54:53.738458    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 19:55:25.498648 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:25.498614    2566 scope.go:117] "RemoveContainer" containerID="e90aac6a501db7ca46acac953b92bde6511e34d556145438e74f8c874073a948"
Jul 23 19:55:29.169127 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.169091    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"]
Jul 23 19:55:29.172753 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.172733    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.176343 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.176307    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-829079\"/\"openshift-service-ca.crt\""
Jul 23 19:55:29.176343 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.176328    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-829079\"/\"auth-test-model-829079-kserve-self-signed-certs\""
Jul 23 19:55:29.176537 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.176312    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-829079\"/\"kube-root-ca.crt\""
Jul 23 19:55:29.176537 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.176307    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-829079\"/\"default-dockercfg-zzhln\""
Jul 23 19:55:29.180658 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.180617    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"]
Jul 23 19:55:29.210846 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.210821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsvw8\" (UniqueName: \"kubernetes.io/projected/84d79f79-a0a4-46f4-950a-73e69bb6d639-kube-api-access-dsvw8\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.210980 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.210855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-home\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.210980 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.210878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-dshm\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.210980 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.210926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-model-cache\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.210980 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.210964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-tmp-dir\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.211137 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.211007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d79f79-a0a4-46f4-950a-73e69bb6d639-tls-certs\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsvw8\" (UniqueName: \"kubernetes.io/projected/84d79f79-a0a4-46f4-950a-73e69bb6d639-kube-api-access-dsvw8\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312163 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-home\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312445 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312233    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-dshm\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312445 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312297    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-model-cache\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312445 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312353    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-tmp-dir\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312445 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312437    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d79f79-a0a4-46f4-950a-73e69bb6d639-tls-certs\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312720 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312700    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-home\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.312821 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.312800    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-model-cache\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.313030 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.313006    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-tmp-dir\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.315100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.315077    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d79f79-a0a4-46f4-950a-73e69bb6d639-tls-certs\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.315100 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.315095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-dshm\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.319924 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.319895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsvw8\" (UniqueName: \"kubernetes.io/projected/84d79f79-a0a4-46f4-950a-73e69bb6d639-kube-api-access-dsvw8\") pod \"auth-test-model-829079-kserve-768d98557f-474hf\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") " pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.485405 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.485321    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:29.619648 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.619624    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"]
Jul 23 19:55:29.621821 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:55:29.621793    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84d79f79_a0a4_46f4_950a_73e69bb6d639.slice/crio-d9070343cec58f2c578ac037392d3e2df7c11387ddcb01193d2215d94fe80bbe WatchSource:0}: Error finding container d9070343cec58f2c578ac037392d3e2df7c11387ddcb01193d2215d94fe80bbe: Status 404 returned error can't find the container with id d9070343cec58f2c578ac037392d3e2df7c11387ddcb01193d2215d94fe80bbe
Jul 23 19:55:29.941582 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.941553    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf" event={"ID":"84d79f79-a0a4-46f4-950a-73e69bb6d639","Type":"ContainerStarted","Data":"bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece"}
Jul 23 19:55:29.941582 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.941591    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf" event={"ID":"84d79f79-a0a4-46f4-950a-73e69bb6d639","Type":"ContainerStarted","Data":"d9070343cec58f2c578ac037392d3e2df7c11387ddcb01193d2215d94fe80bbe"}
Jul 23 19:55:29.958062 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:29.958017    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf" podStartSLOduration=0.958002159 podStartE2EDuration="958.002159ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:55:29.955930278 +0000 UTC m=+2345.244529868" watchObservedRunningTime="2026-07-23 19:55:29.958002159 +0000 UTC m=+2345.246601750"
Jul 23 19:55:39.485845 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:39.485808    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:39.486368 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:39.485896    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:39.498709 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:39.498666    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:55:40.003242 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:55:40.003206    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:57:24.106683 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:24.106631    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"]
Jul 23 19:57:39.127881 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.127805    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf" podUID="84d79f79-a0a4-46f4-950a-73e69bb6d639" containerName="main" containerID="cri-o://bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece" gracePeriod=45
Jul 23 19:57:39.374382 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.374355    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:57:39.443556 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsvw8\" (UniqueName: \"kubernetes.io/projected/84d79f79-a0a4-46f4-950a-73e69bb6d639-kube-api-access-dsvw8\") pod \"84d79f79-a0a4-46f4-950a-73e69bb6d639\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") "
Jul 23 19:57:39.443738 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443611    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-model-cache\") pod \"84d79f79-a0a4-46f4-950a-73e69bb6d639\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") "
Jul 23 19:57:39.443738 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443667    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d79f79-a0a4-46f4-950a-73e69bb6d639-tls-certs\") pod \"84d79f79-a0a4-46f4-950a-73e69bb6d639\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") "
Jul 23 19:57:39.443738 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443709    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-dshm\") pod \"84d79f79-a0a4-46f4-950a-73e69bb6d639\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") "
Jul 23 19:57:39.443902 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443740    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-tmp-dir\") pod \"84d79f79-a0a4-46f4-950a-73e69bb6d639\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") "
Jul 23 19:57:39.443902 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443766    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-home\") pod \"84d79f79-a0a4-46f4-950a-73e69bb6d639\" (UID: \"84d79f79-a0a4-46f4-950a-73e69bb6d639\") "
Jul 23 19:57:39.443902 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.443842    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-model-cache" (OuterVolumeSpecName: "model-cache") pod "84d79f79-a0a4-46f4-950a-73e69bb6d639" (UID: "84d79f79-a0a4-46f4-950a-73e69bb6d639"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:57:39.444058 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.444006    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84d79f79-a0a4-46f4-950a-73e69bb6d639" (UID: "84d79f79-a0a4-46f4-950a-73e69bb6d639"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:57:39.444058 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.444028    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-model-cache\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:57:39.444058 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.444046    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-home" (OuterVolumeSpecName: "home") pod "84d79f79-a0a4-46f4-950a-73e69bb6d639" (UID: "84d79f79-a0a4-46f4-950a-73e69bb6d639"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:57:39.446007 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.445977    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-dshm" (OuterVolumeSpecName: "dshm") pod "84d79f79-a0a4-46f4-950a-73e69bb6d639" (UID: "84d79f79-a0a4-46f4-950a-73e69bb6d639"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:57:39.446007 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.446002    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d79f79-a0a4-46f4-950a-73e69bb6d639-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84d79f79-a0a4-46f4-950a-73e69bb6d639" (UID: "84d79f79-a0a4-46f4-950a-73e69bb6d639"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:57:39.446145 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.446061    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d79f79-a0a4-46f4-950a-73e69bb6d639-kube-api-access-dsvw8" (OuterVolumeSpecName: "kube-api-access-dsvw8") pod "84d79f79-a0a4-46f4-950a-73e69bb6d639" (UID: "84d79f79-a0a4-46f4-950a-73e69bb6d639"). InnerVolumeSpecName "kube-api-access-dsvw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:57:39.544628 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.544597    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d79f79-a0a4-46f4-950a-73e69bb6d639-tls-certs\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:57:39.544628 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.544622    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-dshm\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:57:39.544628 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.544631    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-tmp-dir\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:57:39.544843 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.544638    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84d79f79-a0a4-46f4-950a-73e69bb6d639-home\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:57:39.544843 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.544646    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsvw8\" (UniqueName: \"kubernetes.io/projected/84d79f79-a0a4-46f4-950a-73e69bb6d639-kube-api-access-dsvw8\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:57:39.606448 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.606413    2566 generic.go:356] "Generic (PLEG): container finished" podID="84d79f79-a0a4-46f4-950a-73e69bb6d639" containerID="bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece" exitCode=0
Jul 23 19:57:39.606584 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.606452    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"
Jul 23 19:57:39.606584 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.606504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf" event={"ID":"84d79f79-a0a4-46f4-950a-73e69bb6d639","Type":"ContainerDied","Data":"bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece"}
Jul 23 19:57:39.606584 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.606544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf" event={"ID":"84d79f79-a0a4-46f4-950a-73e69bb6d639","Type":"ContainerDied","Data":"d9070343cec58f2c578ac037392d3e2df7c11387ddcb01193d2215d94fe80bbe"}
Jul 23 19:57:39.606584 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.606562    2566 scope.go:117] "RemoveContainer" containerID="bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece"
Jul 23 19:57:39.616137 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.616118    2566 scope.go:117] "RemoveContainer" containerID="bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece"
Jul 23 19:57:39.616406 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:57:39.616387    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece\": container with ID starting with bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece not found: ID does not exist" containerID="bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece"
Jul 23 19:57:39.616476 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.616415    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece"} err="failed to get container status \"bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece\": rpc error: code = NotFound desc = could not find container \"bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece\": container with ID starting with bc0e8f1bb2f7bbc42382eb82a94b5df602d5ab207bf047e2bed5d16059521ece not found: ID does not exist"
Jul 23 19:57:39.627506 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.627479    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"]
Jul 23 19:57:39.629573 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:39.629552    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-829079/auth-test-model-829079-kserve-768d98557f-474hf"]
Jul 23 19:57:41.238979 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.238943    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d79f79-a0a4-46f4-950a-73e69bb6d639" path="/var/lib/kubelet/pods/84d79f79-a0a4-46f4-950a-73e69bb6d639/volumes"
Jul 23 19:57:41.682863 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.682831    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"]
Jul 23 19:57:41.683257 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.683242    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d79f79-a0a4-46f4-950a-73e69bb6d639" containerName="main"
Jul 23 19:57:41.683393 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.683262    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d79f79-a0a4-46f4-950a-73e69bb6d639" containerName="main"
Jul 23 19:57:41.683393 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.683348    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d79f79-a0a4-46f4-950a-73e69bb6d639" containerName="main"
Jul 23 19:57:41.687917 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.687902    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.690644 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.690625    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-459a5a\"/\"openshift-service-ca.crt\""
Jul 23 19:57:41.691509 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.691494    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-459a5a\"/\"sub-test-model-459a5a-kserve-self-signed-certs\""
Jul 23 19:57:41.691577 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.691501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-459a5a\"/\"default-dockercfg-bjr7g\""
Jul 23 19:57:41.691577 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.691536    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-459a5a\"/\"kube-root-ca.crt\""
Jul 23 19:57:41.695967 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.695943    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"]
Jul 23 19:57:41.765936 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.765899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-home\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.766111 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.765956    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-dshm\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.766111 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.765980    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tmp-dir\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.766111 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.766014    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tls-certs\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.766111 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.766044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-model-cache\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.766317 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.766153    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwmxp\" (UniqueName: \"kubernetes.io/projected/0e4fe546-3fcb-4d48-8f2e-97785c9060da-kube-api-access-cwmxp\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.867575 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.867543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwmxp\" (UniqueName: \"kubernetes.io/projected/0e4fe546-3fcb-4d48-8f2e-97785c9060da-kube-api-access-cwmxp\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.867795 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.867595    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-home\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.867795 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.867632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-dshm\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.867795 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.867655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tmp-dir\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.867795 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.867714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tls-certs\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.867795 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.867746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-model-cache\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.868101 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.868074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-home\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.868101 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.868091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tmp-dir\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.868263 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.868144    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-model-cache\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.870052 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.870025    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-dshm\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.870320 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.870305    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tls-certs\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.875782 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.875763    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwmxp\" (UniqueName: \"kubernetes.io/projected/0e4fe546-3fcb-4d48-8f2e-97785c9060da-kube-api-access-cwmxp\") pod \"sub-test-model-459a5a-kserve-7487bcbfc-jzwrh\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") " pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:41.998404 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:41.998322    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:42.131084 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:42.131051    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"]
Jul 23 19:57:42.134831 ip-10-0-140-14 kubenswrapper[2566]: W0723 19:57:42.134802    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e4fe546_3fcb_4d48_8f2e_97785c9060da.slice/crio-0e08217cfcc9837bc48a5ffef9a6b067c030d5ff60c22b83448e9d3de65b9cc0 WatchSource:0}: Error finding container 0e08217cfcc9837bc48a5ffef9a6b067c030d5ff60c22b83448e9d3de65b9cc0: Status 404 returned error can't find the container with id 0e08217cfcc9837bc48a5ffef9a6b067c030d5ff60c22b83448e9d3de65b9cc0
Jul 23 19:57:42.630476 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:42.630440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh" event={"ID":"0e4fe546-3fcb-4d48-8f2e-97785c9060da","Type":"ContainerStarted","Data":"6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b"}
Jul 23 19:57:42.630854 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:42.630484    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh" event={"ID":"0e4fe546-3fcb-4d48-8f2e-97785c9060da","Type":"ContainerStarted","Data":"0e08217cfcc9837bc48a5ffef9a6b067c030d5ff60c22b83448e9d3de65b9cc0"}
Jul 23 19:57:42.649822 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:42.649772    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh" podStartSLOduration=1.6497577730000001 podStartE2EDuration="1.649757773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:57:42.647710008 +0000 UTC m=+2477.936309595" watchObservedRunningTime="2026-07-23 19:57:42.649757773 +0000 UTC m=+2477.938357393"
Jul 23 19:57:51.999297 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:51.999257    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:51.999705 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:51.999313    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:52.011656 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:52.011631    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:57:52.691613 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:57:52.691582    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:58:42.563041 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:42.563002    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"]
Jul 23 19:58:57.584332 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.584281    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh" podUID="0e4fe546-3fcb-4d48-8f2e-97785c9060da" containerName="main" containerID="cri-o://6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b" gracePeriod=45
Jul 23 19:58:57.833261 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.833238    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:58:57.911199 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911163    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-dshm\") pod \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") "
Jul 23 19:58:57.911376 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911263    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tmp-dir\") pod \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") "
Jul 23 19:58:57.911376 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911303    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-home\") pod \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") "
Jul 23 19:58:57.911376 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911330    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tls-certs\") pod \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") "
Jul 23 19:58:57.911551 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911381    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwmxp\" (UniqueName: \"kubernetes.io/projected/0e4fe546-3fcb-4d48-8f2e-97785c9060da-kube-api-access-cwmxp\") pod \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") "
Jul 23 19:58:57.911551 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911476    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-model-cache\") pod \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\" (UID: \"0e4fe546-3fcb-4d48-8f2e-97785c9060da\") "
Jul 23 19:58:57.911551 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911504    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-home" (OuterVolumeSpecName: "home") pod "0e4fe546-3fcb-4d48-8f2e-97785c9060da" (UID: "0e4fe546-3fcb-4d48-8f2e-97785c9060da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:58:57.911551 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911533    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0e4fe546-3fcb-4d48-8f2e-97785c9060da" (UID: "0e4fe546-3fcb-4d48-8f2e-97785c9060da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:58:57.912082 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911857    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tmp-dir\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:58:57.912082 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911881    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-home\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:58:57.912082 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.911873    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e4fe546-3fcb-4d48-8f2e-97785c9060da" (UID: "0e4fe546-3fcb-4d48-8f2e-97785c9060da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:58:57.913770 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.913739    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-dshm" (OuterVolumeSpecName: "dshm") pod "0e4fe546-3fcb-4d48-8f2e-97785c9060da" (UID: "0e4fe546-3fcb-4d48-8f2e-97785c9060da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 19:58:57.913770 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.913741    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e4fe546-3fcb-4d48-8f2e-97785c9060da-kube-api-access-cwmxp" (OuterVolumeSpecName: "kube-api-access-cwmxp") pod "0e4fe546-3fcb-4d48-8f2e-97785c9060da" (UID: "0e4fe546-3fcb-4d48-8f2e-97785c9060da"). InnerVolumeSpecName "kube-api-access-cwmxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 19:58:57.913894 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:57.913837    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e4fe546-3fcb-4d48-8f2e-97785c9060da" (UID: "0e4fe546-3fcb-4d48-8f2e-97785c9060da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 19:58:58.012416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.012390    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-model-cache\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:58:58.012416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.012414    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e4fe546-3fcb-4d48-8f2e-97785c9060da-dshm\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:58:58.012416 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.012423    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e4fe546-3fcb-4d48-8f2e-97785c9060da-tls-certs\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:58:58.012717 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.012431    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwmxp\" (UniqueName: \"kubernetes.io/projected/0e4fe546-3fcb-4d48-8f2e-97785c9060da-kube-api-access-cwmxp\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 19:58:58.013406 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.013383    2566 generic.go:356] "Generic (PLEG): container finished" podID="0e4fe546-3fcb-4d48-8f2e-97785c9060da" containerID="6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b" exitCode=0
Jul 23 19:58:58.013507 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.013436    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"
Jul 23 19:58:58.013507 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.013473    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh" event={"ID":"0e4fe546-3fcb-4d48-8f2e-97785c9060da","Type":"ContainerDied","Data":"6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b"}
Jul 23 19:58:58.013592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.013514    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh" event={"ID":"0e4fe546-3fcb-4d48-8f2e-97785c9060da","Type":"ContainerDied","Data":"0e08217cfcc9837bc48a5ffef9a6b067c030d5ff60c22b83448e9d3de65b9cc0"}
Jul 23 19:58:58.013592 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.013530    2566 scope.go:117] "RemoveContainer" containerID="6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b"
Jul 23 19:58:58.022895 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.022857    2566 scope.go:117] "RemoveContainer" containerID="6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b"
Jul 23 19:58:58.023196 ip-10-0-140-14 kubenswrapper[2566]: E0723 19:58:58.023172    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b\": container with ID starting with 6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b not found: ID does not exist" containerID="6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b"
Jul 23 19:58:58.023272 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.023208    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b"} err="failed to get container status \"6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b\": rpc error: code = NotFound desc = could not find container \"6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b\": container with ID starting with 6f026d2b34f669a0bb8f7f6336a07e623bf437f1fecde1cc361040446e525c0b not found: ID does not exist"
Jul 23 19:58:58.036758 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.036735    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"]
Jul 23 19:58:58.040231 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:58.040209    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-459a5a/sub-test-model-459a5a-kserve-7487bcbfc-jzwrh"]
Jul 23 19:58:59.239463 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:58:59.239425    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e4fe546-3fcb-4d48-8f2e-97785c9060da" path="/var/lib/kubelet/pods/0e4fe546-3fcb-4d48-8f2e-97785c9060da/volumes"
Jul 23 19:59:25.549081 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:59:25.549049    2566 scope.go:117] "RemoveContainer" containerID="6d61d62eecca1d36d94f3a296424efb6a7b8b1add67dbcf05a7d6f42bfb15f85"
Jul 23 19:59:25.558118 ip-10-0-140-14 kubenswrapper[2566]: I0723 19:59:25.558098    2566 scope.go:117] "RemoveContainer" containerID="7b4ab32001c620df2b69ab4038df1413cf654d37cf37c17d04eb3fb3549c96a2"
Jul 23 20:00:00.132395 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.132314    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"]
Jul 23 20:00:00.132922 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.132900    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e4fe546-3fcb-4d48-8f2e-97785c9060da" containerName="main"
Jul 23 20:00:00.132922 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.132924    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e4fe546-3fcb-4d48-8f2e-97785c9060da" containerName="main"
Jul 23 20:00:00.133178 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.133018    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e4fe546-3fcb-4d48-8f2e-97785c9060da" containerName="main"
Jul 23 20:00:00.136050 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.136033    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"
Jul 23 20:00:00.138331 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.138311    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 20:00:00.145647 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.145618    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"]
Jul 23 20:00:00.147315 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.147299    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"
Jul 23 20:00:00.283725 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.283700    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"]
Jul 23 20:00:00.286173 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:00:00.286144    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f2bc2d_f732_4e56_91b7_a8e64424fe2b.slice/crio-ee520bcfceddfa6369bb80e7cddf5e43d6659f4cbac32c419b6b752e0b761d58 WatchSource:0}: Error finding container ee520bcfceddfa6369bb80e7cddf5e43d6659f4cbac32c419b6b752e0b761d58: Status 404 returned error can't find the container with id ee520bcfceddfa6369bb80e7cddf5e43d6659f4cbac32c419b6b752e0b761d58
Jul 23 20:00:00.336087 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:00.336059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq" event={"ID":"34f2bc2d-f732-4e56-91b7-a8e64424fe2b","Type":"ContainerStarted","Data":"ee520bcfceddfa6369bb80e7cddf5e43d6659f4cbac32c419b6b752e0b761d58"}
Jul 23 20:00:01.342432 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:01.342398    2566 generic.go:356] "Generic (PLEG): container finished" podID="34f2bc2d-f732-4e56-91b7-a8e64424fe2b" containerID="423f956ed546771659cf1a3c12b1b3773246dd7a77a3c135d59bb71fbbfe4505" exitCode=0
Jul 23 20:00:01.342830 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:01.342445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq" event={"ID":"34f2bc2d-f732-4e56-91b7-a8e64424fe2b","Type":"ContainerDied","Data":"423f956ed546771659cf1a3c12b1b3773246dd7a77a3c135d59bb71fbbfe4505"}
Jul 23 20:00:02.480281 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:02.480256    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"
Jul 23 20:00:03.353630 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:03.353595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq" event={"ID":"34f2bc2d-f732-4e56-91b7-a8e64424fe2b","Type":"ContainerDied","Data":"ee520bcfceddfa6369bb80e7cddf5e43d6659f4cbac32c419b6b752e0b761d58"}
Jul 23 20:00:03.353630 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:03.353632    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747280-srgwq"
Jul 23 20:00:03.353873 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:03.353655    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee520bcfceddfa6369bb80e7cddf5e43d6659f4cbac32c419b6b752e0b761d58"
Jul 23 20:00:25.583910 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:25.583879    2566 scope.go:117] "RemoveContainer" containerID="e017922fbe16200d30a8f33c74869252371a1c92ad05b423e966036549f4ba06"
Jul 23 20:00:27.306685 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.306641    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-140495200246832"]
Jul 23 20:00:27.307065 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.307051    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f2bc2d-f732-4e56-91b7-a8e64424fe2b" containerName="cleanup"
Jul 23 20:00:27.307108 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.307067    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f2bc2d-f732-4e56-91b7-a8e64424fe2b" containerName="cleanup"
Jul 23 20:00:27.307169 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.307159    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="34f2bc2d-f732-4e56-91b7-a8e64424fe2b" containerName="cleanup"
Jul 23 20:00:27.311871 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.311848    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:27.314247 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.314227    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gp9k8\""
Jul 23 20:00:27.315258 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.315236    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495200246832"]
Jul 23 20:00:27.472932 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.472903    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj5dh\" (UniqueName: \"kubernetes.io/projected/9afffb3b-21c3-43a0-b329-f01cab7b3249-kube-api-access-jj5dh\") pod \"test-curl-7364-140495200246832\" (UID: \"9afffb3b-21c3-43a0-b329-f01cab7b3249\") " pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:27.573860 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.573765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj5dh\" (UniqueName: \"kubernetes.io/projected/9afffb3b-21c3-43a0-b329-f01cab7b3249-kube-api-access-jj5dh\") pod \"test-curl-7364-140495200246832\" (UID: \"9afffb3b-21c3-43a0-b329-f01cab7b3249\") " pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:27.583265 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.583243    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj5dh\" (UniqueName: \"kubernetes.io/projected/9afffb3b-21c3-43a0-b329-f01cab7b3249-kube-api-access-jj5dh\") pod \"test-curl-7364-140495200246832\" (UID: \"9afffb3b-21c3-43a0-b329-f01cab7b3249\") " pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:27.622292 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.622261    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:27.756584 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.756557    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495200246832"]
Jul 23 20:00:27.758838 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:00:27.758803    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9afffb3b_21c3_43a0_b329_f01cab7b3249.slice/crio-a715f585af55314943761a37bd413523c81d71291bc7985748d7b95a771eca70 WatchSource:0}: Error finding container a715f585af55314943761a37bd413523c81d71291bc7985748d7b95a771eca70: Status 404 returned error can't find the container with id a715f585af55314943761a37bd413523c81d71291bc7985748d7b95a771eca70
Jul 23 20:00:27.760207 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:27.760188    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 20:00:28.483282 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:28.483244    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495200246832" event={"ID":"9afffb3b-21c3-43a0-b329-f01cab7b3249","Type":"ContainerStarted","Data":"a715f585af55314943761a37bd413523c81d71291bc7985748d7b95a771eca70"}
Jul 23 20:00:29.490074 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:29.490035    2566 generic.go:356] "Generic (PLEG): container finished" podID="9afffb3b-21c3-43a0-b329-f01cab7b3249" containerID="abd9121bd4f25744b9a835ff2bd3fb9a543ceac755fc0d9a20bfb35910ef9304" exitCode=0
Jul 23 20:00:29.490467 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:29.490117    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495200246832" event={"ID":"9afffb3b-21c3-43a0-b329-f01cab7b3249","Type":"ContainerDied","Data":"abd9121bd4f25744b9a835ff2bd3fb9a543ceac755fc0d9a20bfb35910ef9304"}
Jul 23 20:00:30.623779 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.623756    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:30.663569 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.663539    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7364-140495200246832_9afffb3b-21c3-43a0-b329-f01cab7b3249/test-curl-7364-140495200246832/0.log"
Jul 23 20:00:30.689727 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.689698    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-140495200246832"]
Jul 23 20:00:30.691806 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.691785    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-140495200246832"]
Jul 23 20:00:30.701956 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.701929    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj5dh\" (UniqueName: \"kubernetes.io/projected/9afffb3b-21c3-43a0-b329-f01cab7b3249-kube-api-access-jj5dh\") pod \"9afffb3b-21c3-43a0-b329-f01cab7b3249\" (UID: \"9afffb3b-21c3-43a0-b329-f01cab7b3249\") "
Jul 23 20:00:30.704068 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.704050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afffb3b-21c3-43a0-b329-f01cab7b3249-kube-api-access-jj5dh" (OuterVolumeSpecName: "kube-api-access-jj5dh") pod "9afffb3b-21c3-43a0-b329-f01cab7b3249" (UID: "9afffb3b-21c3-43a0-b329-f01cab7b3249"). InnerVolumeSpecName "kube-api-access-jj5dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:00:30.803596 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.803521    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj5dh\" (UniqueName: \"kubernetes.io/projected/9afffb3b-21c3-43a0-b329-f01cab7b3249-kube-api-access-jj5dh\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:30.813961 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.810802    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-140495200247344"]
Jul 23 20:00:30.813961 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.811706    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afffb3b-21c3-43a0-b329-f01cab7b3249" containerName="test-curl-7364-140495200246832"
Jul 23 20:00:30.813961 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.811727    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afffb3b-21c3-43a0-b329-f01cab7b3249" containerName="test-curl-7364-140495200246832"
Jul 23 20:00:30.813961 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.811916    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afffb3b-21c3-43a0-b329-f01cab7b3249" containerName="test-curl-7364-140495200246832"
Jul 23 20:00:30.816142 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.816112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:30.817846 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.817822    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495200247344"]
Jul 23 20:00:30.904831 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:30.904796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm4dv\" (UniqueName: \"kubernetes.io/projected/f2288150-fb58-4214-925d-1243990e600a-kube-api-access-cm4dv\") pod \"test-curl-7364-140495200247344\" (UID: \"f2288150-fb58-4214-925d-1243990e600a\") " pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:31.005696 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.005645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4dv\" (UniqueName: \"kubernetes.io/projected/f2288150-fb58-4214-925d-1243990e600a-kube-api-access-cm4dv\") pod \"test-curl-7364-140495200247344\" (UID: \"f2288150-fb58-4214-925d-1243990e600a\") " pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:31.014267 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.014199    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm4dv\" (UniqueName: \"kubernetes.io/projected/f2288150-fb58-4214-925d-1243990e600a-kube-api-access-cm4dv\") pod \"test-curl-7364-140495200247344\" (UID: \"f2288150-fb58-4214-925d-1243990e600a\") " pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:31.126374 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.126308    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:31.240733 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.240703    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afffb3b-21c3-43a0-b329-f01cab7b3249" path="/var/lib/kubelet/pods/9afffb3b-21c3-43a0-b329-f01cab7b3249/volumes"
Jul 23 20:00:31.254337 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.254315    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495200247344"]
Jul 23 20:00:31.256827 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:00:31.256802    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2288150_fb58_4214_925d_1243990e600a.slice/crio-783ee9e49d7227047b5b7675ec0ba4ddd0b2f8eff6c6655eb3c8d923aaef2d8c WatchSource:0}: Error finding container 783ee9e49d7227047b5b7675ec0ba4ddd0b2f8eff6c6655eb3c8d923aaef2d8c: Status 404 returned error can't find the container with id 783ee9e49d7227047b5b7675ec0ba4ddd0b2f8eff6c6655eb3c8d923aaef2d8c
Jul 23 20:00:31.501750 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.501721    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200246832"
Jul 23 20:00:31.501750 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.501736    2566 scope.go:117] "RemoveContainer" containerID="abd9121bd4f25744b9a835ff2bd3fb9a543ceac755fc0d9a20bfb35910ef9304"
Jul 23 20:00:31.503117 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:31.503092    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495200247344" event={"ID":"f2288150-fb58-4214-925d-1243990e600a","Type":"ContainerStarted","Data":"783ee9e49d7227047b5b7675ec0ba4ddd0b2f8eff6c6655eb3c8d923aaef2d8c"}
Jul 23 20:00:32.513303 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:32.513268    2566 generic.go:356] "Generic (PLEG): container finished" podID="f2288150-fb58-4214-925d-1243990e600a" containerID="74106e8a56d01e40f740871c6e846342a8418a098230a9882877dc02859fdecb" exitCode=0
Jul 23 20:00:32.513711 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:32.513345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495200247344" event={"ID":"f2288150-fb58-4214-925d-1243990e600a","Type":"ContainerDied","Data":"74106e8a56d01e40f740871c6e846342a8418a098230a9882877dc02859fdecb"}
Jul 23 20:00:33.652730 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.652703    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:33.671687 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.671653    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7364-140495200247344_f2288150-fb58-4214-925d-1243990e600a/test-curl-7364-140495200247344/0.log"
Jul 23 20:00:33.694169 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.694148    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-140495200247344"]
Jul 23 20:00:33.700081 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.700056    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-140495200247344"]
Jul 23 20:00:33.731534 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.731502    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm4dv\" (UniqueName: \"kubernetes.io/projected/f2288150-fb58-4214-925d-1243990e600a-kube-api-access-cm4dv\") pod \"f2288150-fb58-4214-925d-1243990e600a\" (UID: \"f2288150-fb58-4214-925d-1243990e600a\") "
Jul 23 20:00:33.733791 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.733766    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2288150-fb58-4214-925d-1243990e600a-kube-api-access-cm4dv" (OuterVolumeSpecName: "kube-api-access-cm4dv") pod "f2288150-fb58-4214-925d-1243990e600a" (UID: "f2288150-fb58-4214-925d-1243990e600a"). InnerVolumeSpecName "kube-api-access-cm4dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:00:33.833216 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.833142    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm4dv\" (UniqueName: \"kubernetes.io/projected/f2288150-fb58-4214-925d-1243990e600a-kube-api-access-cm4dv\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:33.855720 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.855688    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:33.856089 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.856077    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2288150-fb58-4214-925d-1243990e600a" containerName="test-curl-7364-140495200247344"
Jul 23 20:00:33.856132 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.856091    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2288150-fb58-4214-925d-1243990e600a" containerName="test-curl-7364-140495200247344"
Jul 23 20:00:33.856185 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.856167    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2288150-fb58-4214-925d-1243990e600a" containerName="test-curl-7364-140495200247344"
Jul 23 20:00:33.859180 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.859163    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:33.868113 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:33.868087    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:34.034821 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.034786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zlgp\" (UniqueName: \"kubernetes.io/projected/7e873edd-5f2b-407c-a574-f7e02643e9f1-kube-api-access-4zlgp\") pod \"test-curl-7364-140495201200720\" (UID: \"7e873edd-5f2b-407c-a574-f7e02643e9f1\") " pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:34.135604 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.135527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zlgp\" (UniqueName: \"kubernetes.io/projected/7e873edd-5f2b-407c-a574-f7e02643e9f1-kube-api-access-4zlgp\") pod \"test-curl-7364-140495201200720\" (UID: \"7e873edd-5f2b-407c-a574-f7e02643e9f1\") " pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:34.143329 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.143304    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zlgp\" (UniqueName: \"kubernetes.io/projected/7e873edd-5f2b-407c-a574-f7e02643e9f1-kube-api-access-4zlgp\") pod \"test-curl-7364-140495201200720\" (UID: \"7e873edd-5f2b-407c-a574-f7e02643e9f1\") " pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:34.169884 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.169860    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:34.299331 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.299307    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:34.302539 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:00:34.302508    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e873edd_5f2b_407c_a574_f7e02643e9f1.slice/crio-6fcbeb69ccb2fe0d25aafc6ff8ee7d1435f2120a49ebd553566c2205f1b1e262 WatchSource:0}: Error finding container 6fcbeb69ccb2fe0d25aafc6ff8ee7d1435f2120a49ebd553566c2205f1b1e262: Status 404 returned error can't find the container with id 6fcbeb69ccb2fe0d25aafc6ff8ee7d1435f2120a49ebd553566c2205f1b1e262
Jul 23 20:00:34.524611 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.524580    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:34.524809 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.524588    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="783ee9e49d7227047b5b7675ec0ba4ddd0b2f8eff6c6655eb3c8d923aaef2d8c"
Jul 23 20:00:34.525788 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:34.525761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495201200720" event={"ID":"7e873edd-5f2b-407c-a574-f7e02643e9f1","Type":"ContainerStarted","Data":"6fcbeb69ccb2fe0d25aafc6ff8ee7d1435f2120a49ebd553566c2205f1b1e262"}
Jul 23 20:00:34.535356 ip-10-0-140-14 kubenswrapper[2566]: E0723 20:00:34.535330    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7364-140495200247344\" is forbidden: User \"system:node:ip-10-0-140-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-14.ec2.internal' and this object" podUID="f2288150-fb58-4214-925d-1243990e600a" pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:35.242526 ip-10-0-140-14 kubenswrapper[2566]: E0723 20:00:35.242477    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7364-140495200247344\" is forbidden: User \"system:node:ip-10-0-140-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-14.ec2.internal' and this object" podUID="f2288150-fb58-4214-925d-1243990e600a" pod="opendatahub/test-curl-7364-140495200247344"
Jul 23 20:00:35.244628 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:35.244600    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2288150-fb58-4214-925d-1243990e600a" path="/var/lib/kubelet/pods/f2288150-fb58-4214-925d-1243990e600a/volumes"
Jul 23 20:00:35.531859 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:35.531772    2566 generic.go:356] "Generic (PLEG): container finished" podID="7e873edd-5f2b-407c-a574-f7e02643e9f1" containerID="b2a28754cbf936db8e94b587c0f7336c1f0569e791bda12b7cc456afc9fb2eb4" exitCode=0
Jul 23 20:00:35.531999 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:35.531857    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495201200720" event={"ID":"7e873edd-5f2b-407c-a574-f7e02643e9f1","Type":"ContainerDied","Data":"b2a28754cbf936db8e94b587c0f7336c1f0569e791bda12b7cc456afc9fb2eb4"}
Jul 23 20:00:36.671793 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.671772    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:36.691139 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.691113    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7364-140495201200720_7e873edd-5f2b-407c-a574-f7e02643e9f1/test-curl-7364-140495201200720/0.log"
Jul 23 20:00:36.711318 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.711293    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:36.713902 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.713882    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:36.758254 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.758224    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zlgp\" (UniqueName: \"kubernetes.io/projected/7e873edd-5f2b-407c-a574-f7e02643e9f1-kube-api-access-4zlgp\") pod \"7e873edd-5f2b-407c-a574-f7e02643e9f1\" (UID: \"7e873edd-5f2b-407c-a574-f7e02643e9f1\") "
Jul 23 20:00:36.760347 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.760317    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e873edd-5f2b-407c-a574-f7e02643e9f1-kube-api-access-4zlgp" (OuterVolumeSpecName: "kube-api-access-4zlgp") pod "7e873edd-5f2b-407c-a574-f7e02643e9f1" (UID: "7e873edd-5f2b-407c-a574-f7e02643e9f1"). InnerVolumeSpecName "kube-api-access-4zlgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:00:36.849017 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.848960    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:36.849352 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.849338    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e873edd-5f2b-407c-a574-f7e02643e9f1" containerName="test-curl-7364-140495201200720"
Jul 23 20:00:36.849406 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.849353    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e873edd-5f2b-407c-a574-f7e02643e9f1" containerName="test-curl-7364-140495201200720"
Jul 23 20:00:36.849445 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.849427    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e873edd-5f2b-407c-a574-f7e02643e9f1" containerName="test-curl-7364-140495201200720"
Jul 23 20:00:36.852410 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.852395    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:36.859085 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.859060    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:36.859175 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.859154    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zlgp\" (UniqueName: \"kubernetes.io/projected/7e873edd-5f2b-407c-a574-f7e02643e9f1-kube-api-access-4zlgp\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:36.960166 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:36.960133    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hbkl\" (UniqueName: \"kubernetes.io/projected/aef39ea5-4b88-445e-bef7-c8caafa39201-kube-api-access-7hbkl\") pod \"test-curl-7364-140495201200720\" (UID: \"aef39ea5-4b88-445e-bef7-c8caafa39201\") " pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:37.060766 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.060733    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hbkl\" (UniqueName: \"kubernetes.io/projected/aef39ea5-4b88-445e-bef7-c8caafa39201-kube-api-access-7hbkl\") pod \"test-curl-7364-140495201200720\" (UID: \"aef39ea5-4b88-445e-bef7-c8caafa39201\") " pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:37.068983 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.068958    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hbkl\" (UniqueName: \"kubernetes.io/projected/aef39ea5-4b88-445e-bef7-c8caafa39201-kube-api-access-7hbkl\") pod \"test-curl-7364-140495201200720\" (UID: \"aef39ea5-4b88-445e-bef7-c8caafa39201\") " pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:37.162047 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.162026    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:37.238761 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.238732    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e873edd-5f2b-407c-a574-f7e02643e9f1" path="/var/lib/kubelet/pods/7e873edd-5f2b-407c-a574-f7e02643e9f1/volumes"
Jul 23 20:00:37.295511 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.295468    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:37.297580 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:00:37.297554    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaef39ea5_4b88_445e_bef7_c8caafa39201.slice/crio-f9a7b8698c7c241f1dbcc8c87b86d0ae463dcaf041dc729361ecf05fd769f535 WatchSource:0}: Error finding container f9a7b8698c7c241f1dbcc8c87b86d0ae463dcaf041dc729361ecf05fd769f535: Status 404 returned error can't find the container with id f9a7b8698c7c241f1dbcc8c87b86d0ae463dcaf041dc729361ecf05fd769f535
Jul 23 20:00:37.542430 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.542348    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:37.542430 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.542375    2566 scope.go:117] "RemoveContainer" containerID="b2a28754cbf936db8e94b587c0f7336c1f0569e791bda12b7cc456afc9fb2eb4"
Jul 23 20:00:37.543981 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:37.543949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495201200720" event={"ID":"aef39ea5-4b88-445e-bef7-c8caafa39201","Type":"ContainerStarted","Data":"f9a7b8698c7c241f1dbcc8c87b86d0ae463dcaf041dc729361ecf05fd769f535"}
Jul 23 20:00:38.552533 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:38.552501    2566 generic.go:356] "Generic (PLEG): container finished" podID="aef39ea5-4b88-445e-bef7-c8caafa39201" containerID="b92674552fcffee15ca8e7259c746b3a521f9833c5b36e1651681bf7a641da52" exitCode=0
Jul 23 20:00:38.553010 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:38.552614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495201200720" event={"ID":"aef39ea5-4b88-445e-bef7-c8caafa39201","Type":"ContainerDied","Data":"b92674552fcffee15ca8e7259c746b3a521f9833c5b36e1651681bf7a641da52"}
Jul 23 20:00:39.692449 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.692421    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:39.713771 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.713750    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7364-140495201200720_aef39ea5-4b88-445e-bef7-c8caafa39201/test-curl-7364-140495201200720/0.log"
Jul 23 20:00:39.741025 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.740998    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:39.743310 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.743286    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-140495201200720"]
Jul 23 20:00:39.788453 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.788430    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hbkl\" (UniqueName: \"kubernetes.io/projected/aef39ea5-4b88-445e-bef7-c8caafa39201-kube-api-access-7hbkl\") pod \"aef39ea5-4b88-445e-bef7-c8caafa39201\" (UID: \"aef39ea5-4b88-445e-bef7-c8caafa39201\") "
Jul 23 20:00:39.790346 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.790321    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef39ea5-4b88-445e-bef7-c8caafa39201-kube-api-access-7hbkl" (OuterVolumeSpecName: "kube-api-access-7hbkl") pod "aef39ea5-4b88-445e-bef7-c8caafa39201" (UID: "aef39ea5-4b88-445e-bef7-c8caafa39201"). InnerVolumeSpecName "kube-api-access-7hbkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:00:39.864960 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.864878    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-140495201201728"]
Jul 23 20:00:39.865310 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.865296    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef39ea5-4b88-445e-bef7-c8caafa39201" containerName="test-curl-7364-140495201200720"
Jul 23 20:00:39.865354 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.865311    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef39ea5-4b88-445e-bef7-c8caafa39201" containerName="test-curl-7364-140495201200720"
Jul 23 20:00:39.865401 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.865380    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aef39ea5-4b88-445e-bef7-c8caafa39201" containerName="test-curl-7364-140495201200720"
Jul 23 20:00:39.868371 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.868356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:39.874006 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.873983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495201201728"]
Jul 23 20:00:39.889777 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.889746    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hbkl\" (UniqueName: \"kubernetes.io/projected/aef39ea5-4b88-445e-bef7-c8caafa39201-kube-api-access-7hbkl\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:39.990581 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:39.990552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkt6j\" (UniqueName: \"kubernetes.io/projected/6980e929-ce67-4f68-827d-f2c27a49f3a0-kube-api-access-wkt6j\") pod \"test-curl-7364-140495201201728\" (UID: \"6980e929-ce67-4f68-827d-f2c27a49f3a0\") " pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:40.092176 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.092141    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkt6j\" (UniqueName: \"kubernetes.io/projected/6980e929-ce67-4f68-827d-f2c27a49f3a0-kube-api-access-wkt6j\") pod \"test-curl-7364-140495201201728\" (UID: \"6980e929-ce67-4f68-827d-f2c27a49f3a0\") " pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:40.100244 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.100217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkt6j\" (UniqueName: \"kubernetes.io/projected/6980e929-ce67-4f68-827d-f2c27a49f3a0-kube-api-access-wkt6j\") pod \"test-curl-7364-140495201201728\" (UID: \"6980e929-ce67-4f68-827d-f2c27a49f3a0\") " pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:40.179179 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.179153    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:40.310377 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.310355    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-140495201201728"]
Jul 23 20:00:40.312233 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:00:40.312209    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6980e929_ce67_4f68_827d_f2c27a49f3a0.slice/crio-54a33d29011ff94ac3ba48c703c6bea51798d18c5190d764ebb82a01ad7dcdbd WatchSource:0}: Error finding container 54a33d29011ff94ac3ba48c703c6bea51798d18c5190d764ebb82a01ad7dcdbd: Status 404 returned error can't find the container with id 54a33d29011ff94ac3ba48c703c6bea51798d18c5190d764ebb82a01ad7dcdbd
Jul 23 20:00:40.564784 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.564661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495201201728" event={"ID":"6980e929-ce67-4f68-827d-f2c27a49f3a0","Type":"ContainerStarted","Data":"54a33d29011ff94ac3ba48c703c6bea51798d18c5190d764ebb82a01ad7dcdbd"}
Jul 23 20:00:40.566008 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.565979    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:40.566131 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:40.566029    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9a7b8698c7c241f1dbcc8c87b86d0ae463dcaf041dc729361ecf05fd769f535"
Jul 23 20:00:40.576489 ip-10-0-140-14 kubenswrapper[2566]: E0723 20:00:40.576465    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7364-140495201200720\" is forbidden: User \"system:node:ip-10-0-140-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-14.ec2.internal' and this object" podUID="aef39ea5-4b88-445e-bef7-c8caafa39201" pod="opendatahub/test-curl-7364-140495201200720"
Jul 23 20:00:41.240202 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:41.240160    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aef39ea5-4b88-445e-bef7-c8caafa39201" path="/var/lib/kubelet/pods/aef39ea5-4b88-445e-bef7-c8caafa39201/volumes"
Jul 23 20:00:41.573828 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:41.573740    2566 generic.go:356] "Generic (PLEG): container finished" podID="6980e929-ce67-4f68-827d-f2c27a49f3a0" containerID="5572f654bf5a7512a22271ca2d540df00b0a61bc7ade072dc58e0837da8ba6d3" exitCode=0
Jul 23 20:00:41.573976 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:41.573826    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-140495201201728" event={"ID":"6980e929-ce67-4f68-827d-f2c27a49f3a0","Type":"ContainerDied","Data":"5572f654bf5a7512a22271ca2d540df00b0a61bc7ade072dc58e0837da8ba6d3"}
Jul 23 20:00:42.708114 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.708089    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:42.727786 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.727762    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7364-140495201201728_6980e929-ce67-4f68-827d-f2c27a49f3a0/test-curl-7364-140495201201728/0.log"
Jul 23 20:00:42.751578 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.751556    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-140495201201728"]
Jul 23 20:00:42.755194 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.755171    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-140495201201728"]
Jul 23 20:00:42.819013 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.818990    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkt6j\" (UniqueName: \"kubernetes.io/projected/6980e929-ce67-4f68-827d-f2c27a49f3a0-kube-api-access-wkt6j\") pod \"6980e929-ce67-4f68-827d-f2c27a49f3a0\" (UID: \"6980e929-ce67-4f68-827d-f2c27a49f3a0\") "
Jul 23 20:00:42.821222 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.821191    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6980e929-ce67-4f68-827d-f2c27a49f3a0-kube-api-access-wkt6j" (OuterVolumeSpecName: "kube-api-access-wkt6j") pod "6980e929-ce67-4f68-827d-f2c27a49f3a0" (UID: "6980e929-ce67-4f68-827d-f2c27a49f3a0"). InnerVolumeSpecName "kube-api-access-wkt6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:00:42.919793 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:42.919757    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkt6j\" (UniqueName: \"kubernetes.io/projected/6980e929-ce67-4f68-827d-f2c27a49f3a0-kube-api-access-wkt6j\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:43.238815 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:43.238744    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6980e929-ce67-4f68-827d-f2c27a49f3a0" path="/var/lib/kubelet/pods/6980e929-ce67-4f68-827d-f2c27a49f3a0/volumes"
Jul 23 20:00:43.586836 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:43.586742    2566 scope.go:117] "RemoveContainer" containerID="5572f654bf5a7512a22271ca2d540df00b0a61bc7ade072dc58e0837da8ba6d3"
Jul 23 20:00:43.587001 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:43.586756    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-140495201201728"
Jul 23 20:00:57.190323 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.190285    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"]
Jul 23 20:00:57.190831 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.190497    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg" podUID="0e5659ba-c43b-46be-a134-d39fb6245aed" containerName="maas-api" containerID="cri-o://0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa" gracePeriod=30
Jul 23 20:00:57.647301 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.647273    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 20:00:57.657335 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.657304    2566 generic.go:356] "Generic (PLEG): container finished" podID="0e5659ba-c43b-46be-a134-d39fb6245aed" containerID="0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa" exitCode=0
Jul 23 20:00:57.657468 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.657344    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"
Jul 23 20:00:57.657468 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.657398    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg" event={"ID":"0e5659ba-c43b-46be-a134-d39fb6245aed","Type":"ContainerDied","Data":"0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa"}
Jul 23 20:00:57.657589 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.657477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg" event={"ID":"0e5659ba-c43b-46be-a134-d39fb6245aed","Type":"ContainerDied","Data":"f53b8850ca866a0fb2af1377589937cf6089f9421eced4d27c77bd93bcb2a1a7"}
Jul 23 20:00:57.657589 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.657501    2566 scope.go:117] "RemoveContainer" containerID="0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa"
Jul 23 20:00:57.667725 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.667697    2566 scope.go:117] "RemoveContainer" containerID="0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa"
Jul 23 20:00:57.668129 ip-10-0-140-14 kubenswrapper[2566]: E0723 20:00:57.668108    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa\": container with ID starting with 0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa not found: ID does not exist" containerID="0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa"
Jul 23 20:00:57.668223 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.668138    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa"} err="failed to get container status \"0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa\": rpc error: code = NotFound desc = could not find container \"0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa\": container with ID starting with 0065ada2419bdfae26b6224dc816212e21822667d1d008cacdf3f87b3f63c7fa not found: ID does not exist"
Jul 23 20:00:57.761178 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.761106    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5659ba-c43b-46be-a134-d39fb6245aed-maas-api-tls\") pod \"0e5659ba-c43b-46be-a134-d39fb6245aed\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") "
Jul 23 20:00:57.761178 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.761154    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5659ba-c43b-46be-a134-d39fb6245aed-openshift-service-ca\") pod \"0e5659ba-c43b-46be-a134-d39fb6245aed\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") "
Jul 23 20:00:57.761364 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.761251    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffcbg\" (UniqueName: \"kubernetes.io/projected/0e5659ba-c43b-46be-a134-d39fb6245aed-kube-api-access-ffcbg\") pod \"0e5659ba-c43b-46be-a134-d39fb6245aed\" (UID: \"0e5659ba-c43b-46be-a134-d39fb6245aed\") "
Jul 23 20:00:57.761451 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.761429    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e5659ba-c43b-46be-a134-d39fb6245aed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e5659ba-c43b-46be-a134-d39fb6245aed" (UID: "0e5659ba-c43b-46be-a134-d39fb6245aed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 20:00:57.761613 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.761590    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5659ba-c43b-46be-a134-d39fb6245aed-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:57.763326 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.763301    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e5659ba-c43b-46be-a134-d39fb6245aed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e5659ba-c43b-46be-a134-d39fb6245aed" (UID: "0e5659ba-c43b-46be-a134-d39fb6245aed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 20:00:57.763407 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.763371    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e5659ba-c43b-46be-a134-d39fb6245aed-kube-api-access-ffcbg" (OuterVolumeSpecName: "kube-api-access-ffcbg") pod "0e5659ba-c43b-46be-a134-d39fb6245aed" (UID: "0e5659ba-c43b-46be-a134-d39fb6245aed"). InnerVolumeSpecName "kube-api-access-ffcbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:00:57.862194 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.862165    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5659ba-c43b-46be-a134-d39fb6245aed-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:57.862194 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.862192    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffcbg\" (UniqueName: \"kubernetes.io/projected/0e5659ba-c43b-46be-a134-d39fb6245aed-kube-api-access-ffcbg\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:00:57.979639 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.979610    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"]
Jul 23 20:00:57.986363 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:57.986339    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-459a5a-7b547cd84-bl8qg"]
Jul 23 20:00:59.238538 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:00:59.238507    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e5659ba-c43b-46be-a134-d39fb6245aed" path="/var/lib/kubelet/pods/0e5659ba-c43b-46be-a134-d39fb6245aed/volumes"
Jul 23 20:01:08.780707 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.780654    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"]
Jul 23 20:01:08.781166 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.781126    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e5659ba-c43b-46be-a134-d39fb6245aed" containerName="maas-api"
Jul 23 20:01:08.781166 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.781139    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e5659ba-c43b-46be-a134-d39fb6245aed" containerName="maas-api"
Jul 23 20:01:08.781166 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.781146    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6980e929-ce67-4f68-827d-f2c27a49f3a0" containerName="test-curl-7364-140495201201728"
Jul 23 20:01:08.781166 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.781156    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6980e929-ce67-4f68-827d-f2c27a49f3a0" containerName="test-curl-7364-140495201201728"
Jul 23 20:01:08.781300 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.781227    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6980e929-ce67-4f68-827d-f2c27a49f3a0" containerName="test-curl-7364-140495201201728"
Jul 23 20:01:08.781300 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.781240    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e5659ba-c43b-46be-a134-d39fb6245aed" containerName="maas-api"
Jul 23 20:01:08.785800 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.785784    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:08.788276 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.788247    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fwrbz\""
Jul 23 20:01:08.791289 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.791270    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"]
Jul 23 20:01:08.866423 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.866389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de27298-5f95-44ce-bb16-85fac267925d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-829079-v4mzq\" (UID: \"0de27298-5f95-44ce-bb16-85fac267925d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:08.967750 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.967714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de27298-5f95-44ce-bb16-85fac267925d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-829079-v4mzq\" (UID: \"0de27298-5f95-44ce-bb16-85fac267925d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:08.968349 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:08.968329    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de27298-5f95-44ce-bb16-85fac267925d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-829079-v4mzq\" (UID: \"0de27298-5f95-44ce-bb16-85fac267925d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:09.097607 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:09.097518    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:09.238534 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:01:09.238504    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de27298_5f95_44ce_bb16_85fac267925d.slice/crio-f09c483c0ff04a94a94b7763eef509313a461fd7a3995545c00387b2b528e254 WatchSource:0}: Error finding container f09c483c0ff04a94a94b7763eef509313a461fd7a3995545c00387b2b528e254: Status 404 returned error can't find the container with id f09c483c0ff04a94a94b7763eef509313a461fd7a3995545c00387b2b528e254
Jul 23 20:01:09.239331 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:09.239311    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"]
Jul 23 20:01:09.723950 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:09.723867    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq" event={"ID":"0de27298-5f95-44ce-bb16-85fac267925d","Type":"ContainerStarted","Data":"9a24348880de869abb7e09864ecf5c53bbe46b2dacbe4d29e6bdcbf6b73c2da7"}
Jul 23 20:01:09.723950 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:09.723920    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq" event={"ID":"0de27298-5f95-44ce-bb16-85fac267925d","Type":"ContainerStarted","Data":"f09c483c0ff04a94a94b7763eef509313a461fd7a3995545c00387b2b528e254"}
Jul 23 20:01:09.752269 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:09.752217    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq" podStartSLOduration=1.752201227 podStartE2EDuration="1.752201227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 20:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:01:09.750592129 +0000 UTC m=+2685.039191730" watchObservedRunningTime="2026-07-23 20:01:09.752201227 +0000 UTC m=+2685.040800818"
Jul 23 20:01:10.730172 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:10.730136    2566 generic.go:356] "Generic (PLEG): container finished" podID="0de27298-5f95-44ce-bb16-85fac267925d" containerID="9a24348880de869abb7e09864ecf5c53bbe46b2dacbe4d29e6bdcbf6b73c2da7" exitCode=0
Jul 23 20:01:10.730594 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:10.730229    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq" event={"ID":"0de27298-5f95-44ce-bb16-85fac267925d","Type":"ContainerDied","Data":"9a24348880de869abb7e09864ecf5c53bbe46b2dacbe4d29e6bdcbf6b73c2da7"}
Jul 23 20:01:11.868630 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:11.868607    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:11.996860 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:11.996767    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de27298-5f95-44ce-bb16-85fac267925d-maas-api-service-ca\") pod \"0de27298-5f95-44ce-bb16-85fac267925d\" (UID: \"0de27298-5f95-44ce-bb16-85fac267925d\") "
Jul 23 20:01:11.997172 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:11.997142    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0de27298-5f95-44ce-bb16-85fac267925d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0de27298-5f95-44ce-bb16-85fac267925d" (UID: "0de27298-5f95-44ce-bb16-85fac267925d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 20:01:12.098244 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:12.098209    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de27298-5f95-44ce-bb16-85fac267925d-maas-api-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:01:12.740807 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:12.740770    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"
Jul 23 20:01:12.740807 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:12.740797    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq" event={"ID":"0de27298-5f95-44ce-bb16-85fac267925d","Type":"ContainerDied","Data":"f09c483c0ff04a94a94b7763eef509313a461fd7a3995545c00387b2b528e254"}
Jul 23 20:01:12.741018 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:12.740841    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f09c483c0ff04a94a94b7763eef509313a461fd7a3995545c00387b2b528e254"
Jul 23 20:01:19.355222 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.355134    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"]
Jul 23 20:01:19.355701 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.355390    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" podUID="2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" containerName="maas-api" containerID="cri-o://762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea" gracePeriod=30
Jul 23 20:01:19.601796 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.601765    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 20:01:19.662612 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.662585    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-openshift-service-ca\") pod \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") "
Jul 23 20:01:19.662780 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.662623    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls\") pod \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") "
Jul 23 20:01:19.662780 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.662662    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5x4j\" (UniqueName: \"kubernetes.io/projected/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-kube-api-access-l5x4j\") pod \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\" (UID: \"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63\") "
Jul 23 20:01:19.663028 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.663003    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" (UID: "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 20:01:19.664958 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.664936    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-kube-api-access-l5x4j" (OuterVolumeSpecName: "kube-api-access-l5x4j") pod "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" (UID: "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63"). InnerVolumeSpecName "kube-api-access-l5x4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 20:01:19.664958 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.664942    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" (UID: "2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 20:01:19.763409 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.763373    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5x4j\" (UniqueName: \"kubernetes.io/projected/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-kube-api-access-l5x4j\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:01:19.763409 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.763400    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-openshift-service-ca\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:01:19.763602 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.763423    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63-maas-api-tls\") on node \"ip-10-0-140-14.ec2.internal\" DevicePath \"\""
Jul 23 20:01:19.781805 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.781775    2566 generic.go:356] "Generic (PLEG): container finished" podID="2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" containerID="762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea" exitCode=0
Jul 23 20:01:19.781940 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.781818    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"
Jul 23 20:01:19.781940 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.781863    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" event={"ID":"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63","Type":"ContainerDied","Data":"762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea"}
Jul 23 20:01:19.782016 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.781941    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6" event={"ID":"2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63","Type":"ContainerDied","Data":"74721c7d24f619be1fbf6871ac2fca7993e8252e61bc233fbbb6eb43e6a80917"}
Jul 23 20:01:19.782016 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.781985    2566 scope.go:117] "RemoveContainer" containerID="762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea"
Jul 23 20:01:19.791345 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.791328    2566 scope.go:117] "RemoveContainer" containerID="762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea"
Jul 23 20:01:19.791649 ip-10-0-140-14 kubenswrapper[2566]: E0723 20:01:19.791630    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea\": container with ID starting with 762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea not found: ID does not exist" containerID="762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea"
Jul 23 20:01:19.791760 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.791655    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea"} err="failed to get container status \"762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea\": rpc error: code = NotFound desc = could not find container \"762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea\": container with ID starting with 762146abd8aed4e7abaa3ae0abbf0ae4a3d628c4c22f78512d178e3d351036ea not found: ID does not exist"
Jul 23 20:01:19.803778 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.803748    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"]
Jul 23 20:01:19.806195 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:19.806170    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-829079-6595dbdf99-qmrq6"]
Jul 23 20:01:21.238475 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:21.238441    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" path="/var/lib/kubelet/pods/2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63/volumes"
Jul 23 20:01:23.871555 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:23.871034    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"]
Jul 23 20:01:23.878004 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:23.877957    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-829079-v4mzq"]
Jul 23 20:01:25.239577 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:25.239546    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de27298-5f95-44ce-bb16-85fac267925d" path="/var/lib/kubelet/pods/0de27298-5f95-44ce-bb16-85fac267925d/volumes"
Jul 23 20:01:36.288960 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:36.288926    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w4xps_377dc391-fe18-445f-9435-064f1a6f47b2/manager/0.log"
Jul 23 20:01:36.746314 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:36.746284    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bf4f8c896-7hpfq_c533b160-80f6-48e1-8978-0d9d4c200aa8/copy-manifests/0.log"
Jul 23 20:01:36.757728 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:36.757699    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bf4f8c896-7hpfq_c533b160-80f6-48e1-8978-0d9d4c200aa8/manager/0.log"
Jul 23 20:01:37.553280 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:37.553237    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747265-95gkw_7f336c71-759e-45a1-94ae-cf8d63a69573/cleanup/0.log"
Jul 23 20:01:37.661209 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:37.661184    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747280-srgwq_34f2bc2d-f732-4e56-91b7-a8e64424fe2b/cleanup/0.log"
Jul 23 20:01:40.579927 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:40.579900    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6474cf8df7-nb642_89d05744-7bc2-4479-928d-4d8e18a829ec/kube-auth-proxy/0.log"
Jul 23 20:01:41.037471 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:41.037421    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wz468_7dd161f0-1d45-467f-b0e6-f513745ba9d9/payload-processing/0.log"
Jul 23 20:01:41.482266 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:41.482243    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9zpww_8b220048-3e22-4d30-8321-2714a6a74dc9/main/0.log"
Jul 23 20:01:41.810589 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:41.810513    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwn7mg_246d10e9-4866-4b65-bfaa-686277c73c03/main/0.log"
Jul 23 20:01:41.916602 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:41.916571    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-sc2pd_6b61ba91-7eeb-4301-8077-ddb7091f726d/main/0.log"
Jul 23 20:01:42.028222 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:42.028179    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-v5z6d_74e72d08-6041-4ed2-bcda-a583049acd4d/main/0.log"
Jul 23 20:01:50.817390 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:50.817364    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5km4c_514184d7-e116-4ec4-a5b9-b2384ddf0468/global-pull-secret-syncer/0.log"
Jul 23 20:01:50.985315 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:50.985276    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ztdvt_d548d642-8e95-4d45-a44a-5cbba4f7565a/konnectivity-agent/0.log"
Jul 23 20:01:51.058137 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:51.058104    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-14.ec2.internal_dbd1baf3c99400a5595a21fea2397e02/haproxy/0.log"
Jul 23 20:01:59.092536 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.092456    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/alertmanager/0.log"
Jul 23 20:01:59.114998 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.114972    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/config-reloader/0.log"
Jul 23 20:01:59.139954 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.139932    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/kube-rbac-proxy-web/0.log"
Jul 23 20:01:59.164463 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.164436    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/kube-rbac-proxy/0.log"
Jul 23 20:01:59.189913 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.189890    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/kube-rbac-proxy-metric/0.log"
Jul 23 20:01:59.211836 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.211806    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/prom-label-proxy/0.log"
Jul 23 20:01:59.234816 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.234792    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df8184d7-bfab-46ef-bc1e-8a9dda541093/init-config-reloader/0.log"
Jul 23 20:01:59.310398 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.310371    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qdmms_f63c15f8-6345-4978-80b7-ec354d2d5718/kube-state-metrics/0.log"
Jul 23 20:01:59.330692 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.330656    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qdmms_f63c15f8-6345-4978-80b7-ec354d2d5718/kube-rbac-proxy-main/0.log"
Jul 23 20:01:59.351687 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.351608    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qdmms_f63c15f8-6345-4978-80b7-ec354d2d5718/kube-rbac-proxy-self/0.log"
Jul 23 20:01:59.383497 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.383462    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f495dc744-qx6s5_fc141d97-e7fd-49f8-b878-5a7eea218ce2/metrics-server/0.log"
Jul 23 20:01:59.410148 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.410124    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-jwckb_9b7569b2-3f22-4015-b8ad-9111642391b4/monitoring-plugin/0.log"
Jul 23 20:01:59.439562 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.439522    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ms8gv_27c57fdc-3ab3-460a-ba96-1215b8c83938/node-exporter/0.log"
Jul 23 20:01:59.459824 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.459798    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ms8gv_27c57fdc-3ab3-460a-ba96-1215b8c83938/kube-rbac-proxy/0.log"
Jul 23 20:01:59.481472 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.481455    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ms8gv_27c57fdc-3ab3-460a-ba96-1215b8c83938/init-textfile/0.log"
Jul 23 20:01:59.676666 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.676635    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-bgkmn_21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c/kube-rbac-proxy-main/0.log"
Jul 23 20:01:59.707415 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.707394    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-bgkmn_21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c/kube-rbac-proxy-self/0.log"
Jul 23 20:01:59.731542 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.731518    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-bgkmn_21b6ce29-a1b2-4a7e-9059-cb3ed4097c4c/openshift-state-metrics/0.log"
Jul 23 20:01:59.772733 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.772712    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/prometheus/0.log"
Jul 23 20:01:59.795356 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.795333    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/config-reloader/0.log"
Jul 23 20:01:59.816334 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.816313    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/thanos-sidecar/0.log"
Jul 23 20:01:59.838432 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.838406    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/kube-rbac-proxy-web/0.log"
Jul 23 20:01:59.860608 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.860591    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/kube-rbac-proxy/0.log"
Jul 23 20:01:59.882867 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.882849    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/kube-rbac-proxy-thanos/0.log"
Jul 23 20:01:59.905323 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.905297    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64a38aba-3f39-4bf7-85f0-da9ab233960e/init-config-reloader/0.log"
Jul 23 20:01:59.935789 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.935726    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-dbrv2_c174cc4d-1aa5-4343-91cb-eaec74dbdc71/prometheus-operator/0.log"
Jul 23 20:01:59.958165 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:01:59.958144    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-dbrv2_c174cc4d-1aa5-4343-91cb-eaec74dbdc71/kube-rbac-proxy/0.log"
Jul 23 20:02:00.014247 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.014221    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-757d74c77c-rk76j_f2ff0639-7b16-49d9-a210-2edce75e9233/telemeter-client/0.log"
Jul 23 20:02:00.070019 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.069988    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-757d74c77c-rk76j_f2ff0639-7b16-49d9-a210-2edce75e9233/reload/0.log"
Jul 23 20:02:00.120265 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.120237    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-757d74c77c-rk76j_f2ff0639-7b16-49d9-a210-2edce75e9233/kube-rbac-proxy/0.log"
Jul 23 20:02:00.184532 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.184507    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cfcccfcf9-dg4l2_063fda30-5178-419b-9ef9-9e8d3d56fbb8/thanos-query/0.log"
Jul 23 20:02:00.207888 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.207812    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cfcccfcf9-dg4l2_063fda30-5178-419b-9ef9-9e8d3d56fbb8/kube-rbac-proxy-web/0.log"
Jul 23 20:02:00.230349 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.230326    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cfcccfcf9-dg4l2_063fda30-5178-419b-9ef9-9e8d3d56fbb8/kube-rbac-proxy/0.log"
Jul 23 20:02:00.253943 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.253917    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cfcccfcf9-dg4l2_063fda30-5178-419b-9ef9-9e8d3d56fbb8/prom-label-proxy/0.log"
Jul 23 20:02:00.276283 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.276253    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cfcccfcf9-dg4l2_063fda30-5178-419b-9ef9-9e8d3d56fbb8/kube-rbac-proxy-rules/0.log"
Jul 23 20:02:00.302872 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.302847    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cfcccfcf9-dg4l2_063fda30-5178-419b-9ef9-9e8d3d56fbb8/kube-rbac-proxy-metrics/0.log"
Jul 23 20:02:00.902631 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.902595    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"]
Jul 23 20:02:00.903047 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.903032    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de27298-5f95-44ce-bb16-85fac267925d" containerName="revoke-keys"
Jul 23 20:02:00.903093 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.903050    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de27298-5f95-44ce-bb16-85fac267925d" containerName="revoke-keys"
Jul 23 20:02:00.903093 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.903073    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" containerName="maas-api"
Jul 23 20:02:00.903093 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.903078    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" containerName="maas-api"
Jul 23 20:02:00.903243 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.903149    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de27298-5f95-44ce-bb16-85fac267925d" containerName="revoke-keys"
Jul 23 20:02:00.903243 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.903160    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b8a4945-34d5-4b6e-8a70-7ee42f4e2d63" containerName="maas-api"
Jul 23 20:02:00.906539 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.906519    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:00.908911 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.908887    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ttxvk\"/\"kube-root-ca.crt\""
Jul 23 20:02:00.909863 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.909843    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ttxvk\"/\"default-dockercfg-nlhhw\""
Jul 23 20:02:00.909964 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.909866    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ttxvk\"/\"openshift-service-ca.crt\""
Jul 23 20:02:00.913986 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:00.913965    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"]
Jul 23 20:02:01.036050 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.036017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr66h\" (UniqueName: \"kubernetes.io/projected/5572087e-0194-45a4-84f8-8341d4417220-kube-api-access-hr66h\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.036209 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.036066    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-podres\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.036209 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.036178    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-sys\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.036209 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.036205    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-lib-modules\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.036328 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.036232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-proc\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137028 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.136992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-sys\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137028 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-lib-modules\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137049    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-proc\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr66h\" (UniqueName: \"kubernetes.io/projected/5572087e-0194-45a4-84f8-8341d4417220-kube-api-access-hr66h\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-podres\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137134    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-sys\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137186    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-lib-modules\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137211    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-proc\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.137521 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.137267    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5572087e-0194-45a4-84f8-8341d4417220-podres\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.146447 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.146423    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr66h\" (UniqueName: \"kubernetes.io/projected/5572087e-0194-45a4-84f8-8341d4417220-kube-api-access-hr66h\") pod \"perf-node-gather-daemonset-vcl9h\" (UID: \"5572087e-0194-45a4-84f8-8341d4417220\") " pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.217613 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.217546    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:01.353803 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.353779    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"]
Jul 23 20:02:01.355879 ip-10-0-140-14 kubenswrapper[2566]: W0723 20:02:01.355839    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5572087e_0194_45a4_84f8_8341d4417220.slice/crio-5e200e7d59ce6dff3b2c6974f45d63542d5960bafe084c2be15c198338f478f8 WatchSource:0}: Error finding container 5e200e7d59ce6dff3b2c6974f45d63542d5960bafe084c2be15c198338f478f8: Status 404 returned error can't find the container with id 5e200e7d59ce6dff3b2c6974f45d63542d5960bafe084c2be15c198338f478f8
Jul 23 20:02:01.987399 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.987363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h" event={"ID":"5572087e-0194-45a4-84f8-8341d4417220","Type":"ContainerStarted","Data":"49b1f19fe2641ecbcd283032dece7e34a38244213fc872e9ab88b50e5fa3330f"}
Jul 23 20:02:01.987623 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.987412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h" event={"ID":"5572087e-0194-45a4-84f8-8341d4417220","Type":"ContainerStarted","Data":"5e200e7d59ce6dff3b2c6974f45d63542d5960bafe084c2be15c198338f478f8"}
Jul 23 20:02:01.987623 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:01.987504    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:02.004339 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:02.004285    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h" podStartSLOduration=2.004269432 podStartE2EDuration="2.004269432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 20:02:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 20:02:02.002607133 +0000 UTC m=+2737.291206723" watchObservedRunningTime="2026-07-23 20:02:02.004269432 +0000 UTC m=+2737.292869023"
Jul 23 20:02:04.395978 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:04.395953    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-946bd_27c16949-2d83-482e-8c4e-0ec67e7a4d38/dns/0.log"
Jul 23 20:02:04.418957 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:04.418932    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-946bd_27c16949-2d83-482e-8c4e-0ec67e7a4d38/kube-rbac-proxy/0.log"
Jul 23 20:02:04.515128 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:04.515103    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8vrb_d4124a71-f788-4b4a-9ac7-388e2eeb2ec7/dns-node-resolver/0.log"
Jul 23 20:02:05.128779 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:05.128748    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-89stp_8f5c1b6f-bc55-49e3-bd1b-bce9051c2e25/node-ca/0.log"
Jul 23 20:02:06.260403 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:06.260366    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6474cf8df7-nb642_89d05744-7bc2-4479-928d-4d8e18a829ec/kube-auth-proxy/0.log"
Jul 23 20:02:06.440712 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:06.440684    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wz468_7dd161f0-1d45-467f-b0e6-f513745ba9d9/payload-processing/0.log"
Jul 23 20:02:07.237369 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:07.237344    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xfq4f_0941cb00-878d-4651-8e88-0512be077273/serve-healthcheck-canary/0.log"
Jul 23 20:02:07.917578 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:07.917553    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9fcz_3d3ecef0-850d-41b2-871a-53da311d0f84/kube-rbac-proxy/0.log"
Jul 23 20:02:07.940657 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:07.940635    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9fcz_3d3ecef0-850d-41b2-871a-53da311d0f84/exporter/0.log"
Jul 23 20:02:07.962708 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:07.962661    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9fcz_3d3ecef0-850d-41b2-871a-53da311d0f84/extractor/0.log"
Jul 23 20:02:08.002109 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:08.002080    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ttxvk/perf-node-gather-daemonset-vcl9h"
Jul 23 20:02:10.541597 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:10.541561    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w4xps_377dc391-fe18-445f-9435-064f1a6f47b2/manager/0.log"
Jul 23 20:02:10.986128 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:10.986035    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bf4f8c896-7hpfq_c533b160-80f6-48e1-8978-0d9d4c200aa8/manager/0.log"
Jul 23 20:02:11.004791 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:11.004761    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bf4f8c896-7hpfq_c533b160-80f6-48e1-8978-0d9d4c200aa8/copy-manifests/0.log"
Jul 23 20:02:12.219131 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:12.219099    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6474cf8df7-nb642_89d05744-7bc2-4479-928d-4d8e18a829ec/kube-auth-proxy/0.log"
Jul 23 20:02:12.368202 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:12.368156    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wz468_7dd161f0-1d45-467f-b0e6-f513745ba9d9/payload-processing/0.log"
Jul 23 20:02:12.389422 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:12.389382    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6fcc6d8b99-nx2w2_c73b206c-bb1d-4cb8-88e9-07f69ba9f9ca/manager/0.log"
Jul 23 20:02:13.692695 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:13.692645    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6474cf8df7-nb642_89d05744-7bc2-4479-928d-4d8e18a829ec/kube-auth-proxy/0.log"
Jul 23 20:02:13.844775 ip-10-0-140-14 kubenswrapper[2566]: I0723 20:02:13.844747    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wz468_7dd161f0-1d45-467f-b0e6-f513745ba9d9/payload-processing/0.log"